Re: Strange metadata being appended in some rows

2018-05-30 Thread Jeff Jirsa
Most of what you describe sounds pretty DSE specific so I suspect your best 
source for answers will be datastax

There were bugs in some versions of Cassandra that caused corruption during a 
few milliseconds race of ALTER TABLE, and if you’re not using compression with 
CRC checking enabled, it’s possible a bad disk or bit flip could have corrupted 
some of your data, but hard to say much beyond that.

-- 
Jeff Jirsa


> On May 30, 2018, at 3:42 PM, Charulata Sharma (charshar)  
> wrote:
> 
> Hi,
>  
> I am observing a very strange behavior in our cluster. Metadata is being 
> prefixed in some rows.
> This metadata cannot be sent by application primarily because application 
> writing to C* will not have this data,
> and also applications use custom Java objects and this metadata doesn’t fall 
> in that category.
>  
> I am suspecting that this is being added during multi data center replication 
> between analytics and transactions cluster.
> Does anyone have any idea on this. I found no clue online also.
>  
> This is the metadata: The nodes mentioned here are the analytics node and the 
> data directory is from DSEFS. We are experiencing
> Some corruption in dsefs and we understood that this is because of the 
> version we are on (DSE 5.1.5), so I suspect this could be related.
>  
> Thanks,
> Charu
>  
>  
> {
> "privateAddress": "333.33.333.333",
> "lastUpdate": "2018-04-25 05:05:40.665+",
> "readOnly": false,
> "up": true,
> "host": "cssdb-prd-07",
> "publicAddress": "333.33.333.333",
> "privatePort": 5599,
> "storageWeight": 1.0,
> "minFreeSpace": 5368709120,
> "publicPort": 5598,
> "dataCenter": "DC1-RPTG",
> "version": 2,
> "locationId": "498ff9f4-a989-461a-8ce7-17c41800",
> "rack": "RACK1",
> "estUsedSpace": 29771,
> "nodeId": "97494bd1-e783-4a8b-b180-a1946defc7cc",
> "estFreeSpace": 726682292224,
> "directory": "/cassandra/data/ccrcprd-cluster/data5/dsefs/data"
>   }


Strange metadata being appended in some rows

2018-05-30 Thread Charulata Sharma (charshar)
Hi,

I am observing a very strange behavior in our cluster. Metadata is being 
prefixed in some rows.
This metadata cannot be sent by application primarily because application 
writing to C* will not have this data,
and also applications use custom Java objects and this metadata doesn’t fall in 
that category.

I am suspecting that this is being added during multi data center replication 
between analytics and transactions cluster.
Does anyone have any idea on this. I found no clue online also.

This is the metadata: The nodes mentioned here are the analytics node and the 
data directory is from DSEFS. We are experiencing
Some corruption in dsefs and we understood that this is because of the version 
we are on (DSE 5.1.5), so I suspect this could be related.

Thanks,
Charu


{
"privateAddress": "333.33.333.333",
"lastUpdate": "2018-04-25 05:05:40.665+",
"readOnly": false,
"up": true,
"host": "cssdb-prd-07",
"publicAddress": "333.33.333.333",
"privatePort": 5599,
"storageWeight": 1.0,
"minFreeSpace": 5368709120,
"publicPort": 5598,
"dataCenter": "DC1-RPTG",
"version": 2,
"locationId": "498ff9f4-a989-461a-8ce7-17c41800",
"rack": "RACK1",
"estUsedSpace": 29771,
"nodeId": "97494bd1-e783-4a8b-b180-a1946defc7cc",
"estFreeSpace": 726682292224,
"directory": "/cassandra/data/ccrcprd-cluster/data5/dsefs/data"
  }