RE: how to store date before > 1970

[adrien ruffie]

Thank Alok,

I checked in Java and Cassandra handle correctly the case.
But in Java following types are need for this behavior:

java.sql.Timestamp
java.math.BigInteger
com.datastax.driver.core.LocalDate (because java.util.Date and java.sql.Date 
not work correctly ...)

Calendar c = new GregorianCalendar(1157, 10, 11);
ttEntity.setCreationDate(new Timestamp(c.getTimeInMillis()));
ttEntity.setEffectiveDate(LocalDate.fromMillisSinceEpoch(c.getTimeInMillis()));
ttEntity.setTimeLong(BigInteger.valueOf(c.getTimeInMillis()));

Thank you for, your reply.

Best regards,

Adrien

De : Alok Dwivedi 
Envoyé : vendredi 12 avril 2019 10:15
À : user@cassandra.apache.org
Objet : Re: how to store date before > 1970

You should be able to use timestamp or date for dates before 1970. Here is an 
example
CREATE TABLE test_ts_date (id int, ts timestamp, dt date, value text, PRIMARY 
KEY (id, ts));
INSERT INTO test_ts_date (id, ts, dt, value) VALUES (1, '2019-04-11 01:20:30', 
'2019-04-11', 'value1');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, '2014-02-12 03:15:30', 
'2014-02-12', 'Value3');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, '1960-02-12 05:25:33', 
'1960-02-12', 'Value1 for ts before 1970');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, -252268040, 
'1890-22-01', 'Value2 for negative ts');

select * from test_ts_date ;

 id | ts  | dt | value
+-++---
  1 |  -252268040 | 1890-01-22 |Value2 for negative ts
  1 | 1960-02-12 05:25:33.00+ | 1960-02-12 | Value1 for ts before 1970
  1 | 2014-02-12 03:15:30.00+ | 2014-02-12 |Value3
  1 | 2019-04-11 01:20:30.00+ | 2019-04-11 |value1


You can choose either of timestamp or date based on what precision you want. 
They both can take values before 1970. I think there might be issue sending 
negative long for timestamps from Java driver (I haven’t tried that) but 
passing dates before 1970 should be fine. Some related tickets for reference.
https://datastax-oss.atlassian.net/browse/JAVA-264
https://datastax-oss.atlassian.net/browse/JAVA-313

Thanks
Alok Dwivedi
Senior Consultant
https://www.instaclustr.com/




On 12 Apr 2019, at 07:25, adrien ruffie 
mailto:adriennolar...@hotmail.fr>> wrote:

Hello all,

I have a tricky question about "how to store a date" if dates can be a date 
prior to 1970 ?
I checked the potential data type and found timestamp and date but both begin 
to the epoch (January 1, 1970) ...

If I want to store oldest dating, which data type and means I can use ?

Thank you and best regards,

Adrien

Re: [EXTERNAL] Re: Getting Consistency level TWO when it is requested LOCAL_ONE

[Jean Carlo]

I think this jira

https://issues.apache.org/jira/browse/CASSANDRA-9895

Answer my question

Saludos

Jean Carlo

"The best way to predict the future is to invent it" Alan Kay


On Fri, Apr 12, 2019 at 10:04 AM Jean Carlo 
wrote:

> Hello Sean
>
> Well this is a little bit confusing. After digging into the doc, I found
> this old documentation of Datastax that says
> "First, we can dynamically adjust behavior depending on the cluster size
> and arrangement. Cassandra prefers to perform batchlog writes to two
> different replicas in the same datacenter as the coordinator."
>
> https://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2
>
> Which may explain the message in the timeout. However I do not know if
> this information is still true. Someone know if this is still true?
>
> Reading the comments in
> https://issues.apache.org/jira/browse/CASSANDRA-9620 that says ' *Writing
> the batch log will always be done using CL ONE.*' Contradict what I
> understood from datastax's doc
>
> Yes I understood batches are not for speed. Still we are using it for a
> consistency need.
>
> @Mahesh Yes we do set the consistency like that
>
> Thank you
>
> Jean Carlo
>
> "The best way to predict the future is to invent it" Alan Kay
>
>
> On Thu, Apr 11, 2019 at 3:39 PM Durity, Sean R <
> sean_r_dur...@homedepot.com> wrote:
>
>> https://issues.apache.org/jira/browse/CASSANDRA-9620 has something
>> similar that was determined to be a driver error. I would start with
>> looking at the driver version and also the RetryPolicy that is in effect
>> for the Cluster. Secondly, I would look at whether a batch is really needed
>> for the statements. Cassandra batches are for atomicity – not speed.
>>
>>
>>
>> Sean Durity
>>
>> Staff Systems Engineer – Cassandra
>>
>> MTC 2250
>>
>> #cassandra - for the latest news and updates
>>
>>
>>
>>
>>
>> *From:* Mahesh Daksha 
>> *Sent:* Thursday, April 11, 2019 5:21 AM
>> *To:* user@cassandra.apache.org
>> *Subject:* [EXTERNAL] Re: Getting Consistency level TWO when it is
>> requested LOCAL_ONE
>>
>>
>>
>> Hi Jean,
>>
>>
>>
>> I want to understand how you are setting the write consistency level as
>> LOCAL ONE. That is with every query you mentioning consistency level or you
>> have set the spring cassandra config with provided consistency level.
>>
>> Like this:
>>
>> cluster.setQueryOptions(new
>> QueryOptions().setConsistencyLevel(ConsistencyLevel.valueOf(cassandraConsistencyLevel)));
>>
>>
>>
>> The only possibility i see of such behavior is its getting overridden
>> from some where.
>>
>>
>>
>> Thanks,
>>
>> Mahesh Daksha
>>
>>
>>
>> On Thu, Apr 11, 2019 at 1:43 PM Jean Carlo 
>> wrote:
>>
>> Hello everyone,
>>
>>
>>
>> I have a case where the developers are using spring data framework for
>> Cassandra. We are writing batches setting consistency level at LOCAL_ONE
>> but we got a timeout like this
>>
>>
>>
>> *Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException:
>> Cassandra timeout during BATCH_LOG write query at consistency TWO (2
>> replica were required but only 1 acknowledged the write)*
>>
>>
>>
>> Is it Cassandra that somehow writes to the *system.batchlog* using
>> consistency TWO or is it spring data that makes some dirty things behind
>> the scenes ?
>>
>> (I want to believe it is the second one)
>>
>>
>>
>> Cheers
>>
>>
>>
>> Jean Carlo
>>
>>
>> "The best way to predict the future is to invent it" Alan Kay
>>
>>
>> --
>>
>> The information in this Internet Email is confidential and may be legally
>> privileged. It is intended solely for the addressee. Access to this Email
>> by anyone else is unauthorized. If you are not the intended recipient, any
>> disclosure, copying, distribution or any action taken or omitted to be
>> taken in reliance on it, is prohibited and may be unlawful. When addressed
>> to our clients any opinions or advice contained in this Email are subject
>> to the terms and conditions expressed in any applicable governing The Home
>> Depot terms of business or client engagement letter. The Home Depot
>> disclaims all responsibility and liability for the accuracy and content of
>> this attachment and for any damages or losses arising from any
>> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
>> items of a destructive nature, which may be contained in this attachment
>> and shall not be liable for direct, indirect, consequential or special
>> damages in connection with this e-mail message or its attachment.
>>
>

Re: how to store date before > 1970

[Alok Dwivedi]

You should be able to use timestamp or date for dates before 1970. Here is an 
example
CREATE TABLE test_ts_date (id int, ts timestamp, dt date, value text, PRIMARY 
KEY (id, ts));  
INSERT INTO test_ts_date (id, ts, dt, value) VALUES (1, '2019-04-11 01:20:30', 
'2019-04-11', 'value1');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, '2014-02-12 03:15:30', 
'2014-02-12', 'Value3');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, '1960-02-12 05:25:33', 
'1960-02-12', 'Value1 for ts before 1970');
INSERT INTO test_ts_date (id, ts, dt, value) VALUES ( 1, -252268040, 
'1890-22-01', 'Value2 for negative ts');

select * from test_ts_date ;

 id | ts  | dt | value
+-++---
  1 |  -252268040 | 1890-01-22 |Value2 for negative ts
  1 | 1960-02-12 05:25:33.00+ | 1960-02-12 | Value1 for ts before 1970
  1 | 2014-02-12 03:15:30.00+ | 2014-02-12 |Value3
  1 | 2019-04-11 01:20:30.00+ | 2019-04-11 |value1


You can choose either of timestamp or date based on what precision you want. 
They both can take values before 1970. I think there might be issue sending 
negative long for timestamps from Java driver (I haven’t tried that) but 
passing dates before 1970 should be fine. Some related tickets for reference. 
https://datastax-oss.atlassian.net/browse/JAVA-264 

https://datastax-oss.atlassian.net/browse/JAVA-313 


Thanks
Alok Dwivedi
Senior Consultant 
https://www.instaclustr.com/




> On 12 Apr 2019, at 07:25, adrien ruffie  wrote:
> 
> Hello all,
> 
> I have a tricky question about "how to store a date" if dates can be a date 
> prior to 1970 ?
> I checked the potential data type and found timestamp and date but both begin 
> to the epoch (January 1, 1970) ...
> 
> If I want to store oldest dating, which data type and means I can use ?
> 
> Thank you and best regards,
> 
> Adrien

Re: Cassandra 2.1.18 - NPE during startup

[Anthony Grasso]

Hi Thomas,

The process you suggested to get around the issue should work with the
system.keyspaces table.

Make sure to backup the original *system.keyspaces* table files on the node
that fails to start. Then, copy only the *system.keyspaces *table files
from a working node into the *system/schema_keyspaces-...* folder of the
node that fails to start.

This method will only work for certain system tables as some of the data
stored in the system tables will differ between nodes.

Regards,
Anthony

On Thu, 28 Mar 2019 at 05:54, Steinmaurer, Thomas <
thomas.steinmau...@dynatrace.com> wrote:

> Hello,
>
>
>
> any ideas regarding below, cause it happened again on a different node.
>
>
>
> Thanks
>
> Thomas
>
>
>
> *From:* Steinmaurer, Thomas 
> *Sent:* Dienstag, 05. Februar 2019 23:03
> *To:* user@cassandra.apache.org
> *Subject:* Cassandra 2.1.18 - NPE during startup
>
>
>
> Hello,
>
>
>
> at a particular customer location, we are seeing the following NPE during
> startup with Cassandra 2.1.18.
>
>
>
> INFO  [SSTableBatchOpen:2] 2019-02-03 13:32:56,131 SSTableReader.java:475
> - Opening
> /var/opt/data/cassandra/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/system-schema_keyspaces-ka-130
> (256 bytes)
>
> ERROR [main] 2019-02-03 13:32:56,552 CassandraDaemon.java:583 - Exception
> encountered during startup
>
> org.apache.cassandra.io.FSReadError: java.lang.NullPointerException
>
> at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:672)
> ~[apache-cassandra-2.1.18.jar:2.1.18]
>
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:310)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:566)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:655)
> [apache-cassandra-2.1.18.jar:2.1.18]
>
> Caused by: java.lang.NullPointerException: null
>
> at
> org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:664)
> ~[apache-cassandra-2.1.18.jar:2.1.18]
>
> ... 3 common frames omitted
>
>
>
> I found https://issues.apache.org/jira/browse/CASSANDRA-10501
> ,
> but this should be fixed in 2.1.18.
>
>
>
> Is the above log stating that it is caused by a system keyspace related
> SSTable?
>
>
>
> This is a 3 node setup with 2 others running fine. If system table related
> and as LocalStrategy is used as replication strategy (to my knowledge),
> perhaps simply copying over data for the schema_keyspaces table from
> another node might fix it?
>
>
>
> Any help appreciated.
>
>
>
> Thanks.
>
> Thomas
>
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
> The contents of this e-mail are intended for the named addressee only. It
> contains information that may be confidential. Unless you are the named
> addressee or an authorized designee, you may not copy or use it, or
> disclose it to anyone else. If you received it in error please notify us
> immediately and then destroy it. Dynatrace Austria GmbH (registration
> number FN 91482h) is a company registered in Linz whose registered office
> is at 4040 Linz, Austria, Freistädterstraße 313
>

Re: [EXTERNAL] Re: Getting Consistency level TWO when it is requested LOCAL_ONE

[Jean Carlo]

Hello Sean

Well this is a little bit confusing. After digging into the doc, I found
this old documentation of Datastax that says
"First, we can dynamically adjust behavior depending on the cluster size
and arrangement. Cassandra prefers to perform batchlog writes to two
different replicas in the same datacenter as the coordinator."

https://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2

Which may explain the message in the timeout. However I do not know if this
information is still true. Someone know if this is still true?

Reading the comments in https://issues.apache.org/jira/browse/CASSANDRA-9620
that says ' *Writing the batch log will always be done using CL ONE.*'
Contradict what I understood from datastax's doc

Yes I understood batches are not for speed. Still we are using it for a
consistency need.

@Mahesh Yes we do set the consistency like that

Thank you

Jean Carlo

"The best way to predict the future is to invent it" Alan Kay


On Thu, Apr 11, 2019 at 3:39 PM Durity, Sean R 
wrote:

> https://issues.apache.org/jira/browse/CASSANDRA-9620 has something
> similar that was determined to be a driver error. I would start with
> looking at the driver version and also the RetryPolicy that is in effect
> for the Cluster. Secondly, I would look at whether a batch is really needed
> for the statements. Cassandra batches are for atomicity – not speed.
>
>
>
> Sean Durity
>
> Staff Systems Engineer – Cassandra
>
> MTC 2250
>
> #cassandra - for the latest news and updates
>
>
>
>
>
> *From:* Mahesh Daksha 
> *Sent:* Thursday, April 11, 2019 5:21 AM
> *To:* user@cassandra.apache.org
> *Subject:* [EXTERNAL] Re: Getting Consistency level TWO when it is
> requested LOCAL_ONE
>
>
>
> Hi Jean,
>
>
>
> I want to understand how you are setting the write consistency level as
> LOCAL ONE. That is with every query you mentioning consistency level or you
> have set the spring cassandra config with provided consistency level.
>
> Like this:
>
> cluster.setQueryOptions(new
> QueryOptions().setConsistencyLevel(ConsistencyLevel.valueOf(cassandraConsistencyLevel)));
>
>
>
> The only possibility i see of such behavior is its getting overridden from
> some where.
>
>
>
> Thanks,
>
> Mahesh Daksha
>
>
>
> On Thu, Apr 11, 2019 at 1:43 PM Jean Carlo 
> wrote:
>
> Hello everyone,
>
>
>
> I have a case where the developers are using spring data framework for
> Cassandra. We are writing batches setting consistency level at LOCAL_ONE
> but we got a timeout like this
>
>
>
> *Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException:
> Cassandra timeout during BATCH_LOG write query at consistency TWO (2
> replica were required but only 1 acknowledged the write)*
>
>
>
> Is it Cassandra that somehow writes to the *system.batchlog* using
> consistency TWO or is it spring data that makes some dirty things behind
> the scenes ?
>
> (I want to believe it is the second one)
>
>
>
> Cheers
>
>
>
> Jean Carlo
>
>
> "The best way to predict the future is to invent it" Alan Kay
>
>
> --
>
> The information in this Internet Email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this Email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful. When addressed
> to our clients any opinions or advice contained in this Email are subject
> to the terms and conditions expressed in any applicable governing The Home
> Depot terms of business or client engagement letter. The Home Depot
> disclaims all responsibility and liability for the accuracy and content of
> this attachment and for any damages or losses arising from any
> inaccuracies, errors, viruses, e.g., worms, trojan horses, etc., or other
> items of a destructive nature, which may be contained in this attachment
> and shall not be liable for direct, indirect, consequential or special
> damages in connection with this e-mail message or its attachment.
>

TR: how to store date before > 1970

[adrien ruffie]

Hello all,

I have a tricky question about "how to store a date" if dates can be a date 
prior to 1970 ?
I checked the potential data type and found timestamp and date but both begin 
to the epoch (January 1, 1970) ...

If I want to store oldest dating, which data type and means I can use ?

Thank you and best regards,

Adrien