David Schlosnagle created CASSANDRA-10433:
---------------------------------------------

             Summary: Reduce contention in CompositeType instance interning
                 Key: CASSANDRA-10433
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10433
             Project: Cassandra
          Issue Type: Improvement
         Environment: Cassandra 2.2.1 running on 6 AWS c3.4xlarge nodes, CentOS 
6.6
            Reporter: David Schlosnagle
            Priority: Minor


While running some workload tests on Cassandra 2.2.1 and profiling with flight 
recorder in a test environment, we have noticed significant contention on the 
static synchronized 
org.apache.cassandra.db.marshal.CompositeType.getInstance(List) method.

We are seeing threads blocked for 22.828 seconds from a 60 second snapshot 
while under a mix of reads and writes from a Thrift based client.

I would propose to reduce contention in 
org.apache.cassandra.db.marshal.CompositeType.getInstance(List) by using a 
ConcurrentHashMap for the instances cache.

{code}
Contention Back Trace
org.apache.cassandra.db.marshal.CompositeType.getInstance(List)
  
org.apache.cassandra.db.composites.AbstractCompoundCellNameType.asAbstractType()
    org.apache.cassandra.db.SuperColumns.getComparatorFor(CFMetaData, boolean)
      org.apache.cassandra.db.SuperColumns.getComparatorFor(CFMetaData, 
ByteBuffer)
        
org.apache.cassandra.thrift.ThriftValidation.validateColumnNames(CFMetaData, 
ByteBuffer, Iterable)
          
org.apache.cassandra.thrift.ThriftValidation.validateColumnPath(CFMetaData, 
ColumnPath)
            
org.apache.cassandra.thrift.ThriftValidation.validateColumnOrSuperColumn(CFMetaData,
 ByteBuffer, ColumnOrSuperColumn)
              
org.apache.cassandra.thrift.ThriftValidation.validateMutation(CFMetaData, 
ByteBuffer, Mutation)
                
org.apache.cassandra.thrift.CassandraServer.createMutationList(ConsistencyLevel,
 Map, boolean)
                  org.apache.cassandra.thrift.CassandraServer.batch_mutate(Map, 
ConsistencyLevel)
                    
org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.getResult(Cassandra$Iface,
 Cassandra$batch_mutate_args)
        org.apache.cassandra.thrift.ThriftValidation.validateRange(CFMetaData, 
ColumnParent, SliceRange)
          
org.apache.cassandra.thrift.ThriftValidation.validatePredicate(CFMetaData, 
ColumnParent, SlicePredicate)
            
org.apache.cassandra.thrift.CassandraServer.get_range_slices(ColumnParent, 
SlicePredicate, KeyRange, ConsistencyLevel)
              
org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Cassandra$Iface,
 Cassandra$get_range_slices_args)
                
org.apache.cassandra.thrift.Cassandra$Processor$get_range_slices.getResult(Object,
 TBase)
                  org.apache.thrift.ProcessFunction.process(int, TProtocol, 
TProtocol, Object)
                    org.apache.thrift.TBaseProcessor.process(TProtocol, 
TProtocol)
                      
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run()
                        
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor$Worker)
                          java.util.concurrent.ThreadPoolExecutor$Worker.run()
            
org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(String, List, 
ColumnParent, long, SlicePredicate, ConsistencyLevel, ClientState)
              org.apache.cassandra.thrift.CassandraServer.multiget_slice(List, 
ColumnParent, SlicePredicate, ConsistencyLevel)
                
org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Cassandra$Iface,
 Cassandra$multiget_slice_args)
                  
org.apache.cassandra.thrift.Cassandra$Processor$multiget_slice.getResult(Object,
 TBase)
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to