Re: write latency on single partition table

[onmstester onmstester]

The size is less than 50MB


Sent using Zoho Mail






 On Sat, 07 Apr 2018 09:09:41 +0430 Laxmikant Upadhyay 
laxmikant@gmail.com wrote 




It seems your partition size is more..what is the size of value field ? Try to 
keep your partition size within 100 mb.



On Sat, Apr 7, 2018, 9:45 AM onmstester onmstester onmstes...@zoho.com 
wrote:









I've defained a table like this



create table test (

hours int,

key1 int,

value1 varchar,

primary key (hours,key1)

)



For one hour every input would be written in single partition, because i need 
to group by some 500K records in the partition for a report with expected 
response time in less than 1 seconds so using key1 in partition key would made 
500K partitions which would be slow on reads.

Although using  this mechanism gains  1 seconds response time on reads but 
the write delay increased surprisingly, for this table write latency reported 
by cfstats is more than 100ms but for other tables which accessing thousands of 
partitions while writing in 1 hour , the write delay is 0.02ms. But i was 
expecting that writes to test table be faster than other tables because always 
only one node and one partition would be accessed, so no memtable switch 
happens and all writes would be local to a single node?!

Should i add another key to my partition key to distribute data on all of nodes?



Sent using Zoho Mail

Re: write latency on single partition table

[Laxmikant Upadhyay]

It seems your partition size is more..what is the size of value field ? Try
to keep your partition size within 100 mb.

On Sat, Apr 7, 2018, 9:45 AM onmstester onmstester 
wrote:

>
> I've defained a table like this
>
> create table test (
> hours int,
> key1 int,
> value1 varchar,
> primary key (hours,key1)
> )
>
> For one hour every input would be written in single partition, because i
> need to group by some 500K records in the partition for a report with
> expected response time in less than 1 seconds so using key1 in partition
> key would made 500K partitions which would be slow on reads.
> Although using  this mechanism gains < 1 seconds response time on reads
> but the write delay increased surprisingly, for this table write latency
> reported by cfstats is more than 100ms but for other tables which accessing
> thousands of partitions while writing in 1 hour , the write delay is
> 0.02ms. But i was expecting that writes to test table be faster than other
> tables because always only one node and one partition would be accessed, so
> no memtable switch happens and all writes would be local to a single node?!
> Should i add another key to my partition key to distribute data on all of
> nodes?
>
> Sent using Zoho Mail 
>
>
>

write latency on single partition table

[onmstester onmstester]

I've defained a table like this



create table test (

hours int,

key1 int,

value1 varchar,

primary key (hours,key1)
)



For one hour every input would be written in single partition, because i need 
to group by some 500K records in the partition for a report with expected 
response time in less than 1 seconds so using key1 in partition key would made 
500K partitions which would be slow on reads.

Although using  this mechanism gains  1 seconds response time on reads but 
the write delay increased surprisingly, for this table write latency reported 
by cfstats is more than 100ms but for other tables which accessing thousands of 
partitions while writing in 1 hour , the write delay is 0.02ms. But i was 
expecting that writes to test table be faster than other tables because always 
only one node and one partition would be accessed, so no memtable switch 
happens and all writes would be local to a single node?!

Should i add another key to my partition key to distribute data on all of nodes?


Sent using Zoho Mail

Re: How to restrict users to specific DC.

[Pranay akula]

Thanks Jim for your reply,

> ... just in case even he change his contact points he shouldn't be able
execute queries on DC2.

 What I mean was we have 2 DC only 1 serving traffic, Let's say an
individual user wants to run a query from cqlsh/Devcenter on DC serving
requests, I want to prevent it.

I kind of think it's not possible but wanted to know if there were ways to
implement it like killing the session or similar

Thanks
Pranay

On Fri, Apr 6, 2018, 12:22 PM Jim Witschey 
wrote:

> Pranay,
>
> > Is it possible to restrict users to specific DC in cassandra,  let's say
> an user A is connecting to DC1 and executing queries, how to can I restrict
> that user to that particular DC...
>
> This part sounds like a job for a DC-aware load-balancing policy in the
> driver.
>
> > ... just in case even he change his contact points he shouldn't be able
> execute queries on DC2.
>
> This part confuses me. What problem are you trying to solve? Are you
> concerned about a DC getting hit with more requests than you'd like
> because of a misconfigured driver?
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>

Re: How to restrict users to specific DC.

[Jim Witschey]

Pranay,

> Is it possible to restrict users to specific DC in cassandra,  let's say an 
> user A is connecting to DC1 and executing queries, how to can I restrict that 
> user to that particular DC...

This part sounds like a job for a DC-aware load-balancing policy in the driver.

> ... just in case even he change his contact points he shouldn't be able 
> execute queries on DC2.

This part confuses me. What problem are you trying to solve? Are you
concerned about a DC getting hit with more requests than you'd like
because of a misconfigured driver?

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

How to restrict users to specific DC.

[Pranay akula]

Hi all,

Is it possible to restrict users to specific DC in cassandra,  let's say an
user A is connecting to DC1 and executing queries, how to can I restrict
that user to that particular DC, just in case even he change his contact
points he shouldn't be able execute queries on DC2.

Is it something feasible from cassandra end ?? ( I couldn't think of any )


Thanks
Pranay.

Re: tablestats and gossip

[Chris Lohfink]

Yes, its the count of all locally applied writes to that table. A insert to a 
table with a RF=3 should increase the local write count by 1 on 3 different 
nodes.

Chris

> On Apr 6, 2018, at 5:00 AM, Grzegorz Pietrusza  wrote:
> 
> Hi all
> 
> Does local write count provided by tablestats include writes from gossip?


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Cassandra Table "Rename"

[Nitan Kainth]

No, it’s a standard method for renaming tables/kespaces.

For Cassandra datafiles can be renamed, as long as table definition and path is 
correct and version is same.

Sent from my iPhone

> On Apr 6, 2018, at 5:58 AM, Marcel Villet  wrote:
> 
> Hi All
>  
> I did an experiment whereby I recreated a table but with a different name and 
> copied the original sstable files to the sstable directory of the new table.
>  
> When I queried the new table the old data was shown.
>  
> So it looks like this trick worked.
>  
> Are there any caveats with copying data this way?
>  
> Thanks
> Marcel Villet 
> Osprey Pro Team Leader and Lead Architect
> CT LAB (PTY) LTD - Bringing Quality To Power
> T  +27 (0)21-880-9915  |   F  +27 (0)21-880-1088  |  www.ctlab.com  |  
> www.pq-portal.com  
> 15 Termo Lane, Techno Park, Stellenbosch, South Africa, 7600  |  PO Box 897, 
> Stellenbosch,7599.
> 
>  
> 
> 
> This email and any files transmitted with it are confidential and intended 
> solely for the use of the individual or entity to whom they are addressed. If 
> you have received this email in error please notify the system administrator.

Cassandra Table "Rename"

[Marcel Villet]

Hi All

I did an experiment whereby I recreated a table but with a different name and 
copied the original sstable files to the sstable directory of the new table.

When I queried the new table the old data was shown.

So it looks like this trick worked.

Are there any caveats with copying data this way?

Thanks
Marcel Villet
Osprey Pro Team Leader and Lead Architect
CT LAB (PTY) LTD - Bringing Quality To Power
T  +27 (0)21-880-9915  |   F  +27 (0)21-880-1088  |  
www.ctlab.com  |  
www.pq-portal.com
15 Termo Lane, Techno Park, Stellenbosch, South Africa, 7600  |  PO Box 897, 
Stellenbosch,7599.
[cid:E9CDFBB1-258E-48CA-8E20-3ED934168E78]





This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. If 
you have received this email in error please notify the system 
manager:administra...@ctlab.com

tablestats and gossip

[Grzegorz Pietrusza]

Hi all

Does local write count provided by tablestats include writes from gossip?

Re: Shifting data to DCOS

[kurt greaves]

Without looking at the code I'd say maybe the keyspaces are displayed
purely because the directories exist (but it seems unlikely). The process
you should follow instead is to exclude the system keyspaces for each node
and manually apply your schema, then upload your CFs into the correct
directory. Note this only works when RF=#nodes, if you have more nodes you
need to take tokens into account when restoring.

On Fri., 6 Apr. 2018, 17:16 Affan Syed,  wrote:

> Michael,
>
> both of the folders are with hash, so I dont think that would be an issue.
>
> What is strange is why the tables dont show up if the keyspaces are
> visible. Shouldnt that be a meta data that can be edited once and then be
> visible?
>
> Affan
>
> - Affan
>
> On Thu, Apr 5, 2018 at 7:55 PM, Michael Shuler 
> wrote:
>
>> On 04/05/2018 09:04 AM, Faraz Mateen wrote:
>> >
>> > For example,  if the table is *data_main_bim_dn_10*, its data directory
>> > is named data_main_bim_dn_10-a73202c02bf311e8b5106b13f463f8b9. I created
>> > a new table with the same name through cqlsh. This resulted in creation
>> > of another directory with a different hash i.e.
>> > data_main_bim_dn_10-c146e8d038c611e8b48cb7bc120612c9. I copied all data
>> > from the former to the latter.
>> >
>> > Then I ran *"nodetool refresh ks1  data_main_bim_dn_10"*. After that I
>> > was able to access all data contents through cqlsh.
>> >
>> > Now, the problem is, I have around 500 tables and the method I mentioned
>> > above is quite cumbersome. Bulkloading through sstableloader or remote
>> > seeding are also a couple of options but they will take a lot of time.
>> > Does anyone know an easier way to shift all my data to new setup on
>> DC/OS?
>>
>> For upgrade support from older versions of C* that did not have the hash
>> on the data directory, the table data dir can be just
>> `data_main_bim_dn_10` without the appended hash, as in your example.
>>
>> Give that a quick test to see if that simplifies things for you.
>>
>> --
>> Kind regards,
>> Michael
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: user-h...@cassandra.apache.org
>>
>>
>

Re: Upgrade to 3.11.2 disabled JMX

[Nicolas Guyomar]

Hi Lucas,

There are usually some logs in system.log at node startup regarding JMX
initialization, are those OK ?

On 5 April 2018 at 22:13, Lucas Benevides 
wrote:

> Dear community members,
>
> I have just upgraded my Cassandra from version 3.11.1 to 3.11.2. I kept my
> previous configuration files: cassandra.yaml and cassandra-env.sh. However,
> when I started the cassandra service, I couldn't connect via JMX (tried to
> to it with a java program, with JConsole and a prometheus client).
>
> When I run netstat -na it does not show port 7199 open.
> Tried to look at the logs but didn't see anything.
>
> Can you figure out why it happened and point any possible solution? Config
> files enable JMX with authtenticaion=false, but it doesn't work.
>
> Thanks in advance,
> Lucas Benevides
>

Re: Shifting data to DCOS

[Affan Syed]

Michael,

both of the folders are with hash, so I dont think that would be an issue.

What is strange is why the tables dont show up if the keyspaces are
visible. Shouldnt that be a meta data that can be edited once and then be
visible?

Affan

- Affan

On Thu, Apr 5, 2018 at 7:55 PM, Michael Shuler 
wrote:

> On 04/05/2018 09:04 AM, Faraz Mateen wrote:
> >
> > For example,  if the table is *data_main_bim_dn_10*, its data directory
> > is named data_main_bim_dn_10-a73202c02bf311e8b5106b13f463f8b9. I created
> > a new table with the same name through cqlsh. This resulted in creation
> > of another directory with a different hash i.e.
> > data_main_bim_dn_10-c146e8d038c611e8b48cb7bc120612c9. I copied all data
> > from the former to the latter.
> >
> > Then I ran *"nodetool refresh ks1  data_main_bim_dn_10"*. After that I
> > was able to access all data contents through cqlsh.
> >
> > Now, the problem is, I have around 500 tables and the method I mentioned
> > above is quite cumbersome. Bulkloading through sstableloader or remote
> > seeding are also a couple of options but they will take a lot of time.
> > Does anyone know an easier way to shift all my data to new setup on
> DC/OS?
>
> For upgrade support from older versions of C* that did not have the hash
> on the data directory, the table data dir can be just
> `data_main_bim_dn_10` without the appended hash, as in your example.
>
> Give that a quick test to see if that simplifies things for you.
>
> --
> Kind regards,
> Michael
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>