[
https://issues.apache.org/jira/browse/CASSANDRA-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022358#comment-15022358
]
Ariel Weisberg commented on CASSANDRA-10592:
--------------------------------------------
bq. The new property seems like something no one will ever know about or use.
Do we need it or is it an over optimization?
It's not there to allow optimization. It's there in case we made a bad choice
for the constant in some scenario and people in the field need to be able to
fix it without changing code. If it was something that people should actually
be changing it would be in the YAML as a documented tunable.
bq. BDO.doFlush does nothing with count
BDO doesn't need the count to flush. BDO flushes to a channel which doesn't
have any use for that information.
bq. Why does flush call reallocate? If that's just a hack to make sure we
always reallocate, we've now made it worse by passing a param only used
by reallocate and not flush see comment above.
I don't follow how things are made worse. The count is just a hint to make sure
that the resize is big enough, but resizing still takes place at 0 the way it
used to.
Users of the class causing reallocation via {{flush()}} is yucky and yeah we
shouldn't allow that since I don't see how people can benefit from it, and they
can be hurt by calling it accidentally.
bq. Why does reallocate always reallocate even if you send in 0
I will use <= 0 to indicate that no resizing should occur in DataOutputBuffer
like you are suggesting.
> IllegalArgumentException in DataOutputBuffer.reallocate
> -------------------------------------------------------
>
> Key: CASSANDRA-10592
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10592
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction, Local Write-Read Paths, Streaming and
> Messaging
> Reporter: Sebastian Estevez
> Assignee: Ariel Weisberg
> Fix For: 3.0.1, 3.1, 2.2.x
>
>
> CORRECTION-
> It turns out the exception occurs when running a read using a thrift jdbc
> driver. Once you have loaded the data with stress below, run
> SELECT * FROM "autogeneratedtest"."transaction_by_retailer" using this tool -
> http://www.aquafold.com/aquadatastudio_downloads.html
>
> The exception:
> {code}
> WARN [SharedPool-Worker-1] 2015-10-22 12:58:20,792
> AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.RuntimeException: java.lang.IllegalArgumentException
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2366)
> ~[main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_60]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [main/:na]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> Caused by: java.lang.IllegalArgumentException: null
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:334) ~[na:1.8.0_60]
> at
> org.apache.cassandra.io.util.DataOutputBuffer.reallocate(DataOutputBuffer.java:63)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.DataOutputBuffer.doFlush(DataOutputBuffer.java:57)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:132)
> ~[main/:na]
> at
> org.apache.cassandra.io.util.BufferedDataOutputStreamPlus.write(BufferedDataOutputStreamPlus.java:151)
> ~[main/:na]
> at
> org.apache.cassandra.utils.ByteBufferUtil.writeWithVIntLength(ByteBufferUtil.java:296)
> ~[main/:na]
> at
> org.apache.cassandra.db.marshal.AbstractType.writeValue(AbstractType.java:374)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.BufferCell$Serializer.serialize(BufferCell.java:263)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:183)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:108)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:96)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> ~[main/:na]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> ~[main/:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:381)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:136)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:128)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.<init>(ReadResponse.java:123)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65)
> ~[main/:na]
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:289)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1697)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2362)
> ~[main/:na]
> ... 4 common frames omitted
> {code}
> I was running this command:
> {code}
> tools/bin/cassandra-stress user
> profile=~/Desktop/startup/stress/stress.yaml n=100000 ops\(insert=1\) -rate
> threads=30
> {code}
> Here's the stress.yaml UPDATED!
> {code}
> ### DML ### THIS IS UNDER CONSTRUCTION!!!
> # Keyspace Name
> keyspace: autogeneratedtest
> # The CQL for creating a keyspace (optional if it already exists)
> keyspace_definition: |
> CREATE KEYSPACE autogeneratedtest WITH replication = {'class':
> 'SimpleStrategy', 'replication_factor': 1};
> # Table name
> table: test
> # The CQL for creating a table you wish to stress (optional if it already
> exists)
> table_definition:
> CREATE TABLE test (
> a int,
> b int,
> c int,
> d int,
> e int,
> f timestamp,
> g text,
> h bigint,
> i text,
> j text,
> k bigint,
> l text,
> m text,
> n float,
> o int,
> p float,
> q float,
> r text,
> s float,
> PRIMARY KEY ((a, c, d, b, e), m, f, g)
> );
> ### Column Distribution Specifications ###
> columnspec:
> - name: a
> size: uniform(4..4)
> population: uniform(1..500)
> - name: b
> size: uniform(4..4)
> population: uniform(2..3000)
> - name: c
> size: uniform(4..4)
> population: uniform(1..100)
> - name: d
> size: uniform(4..4)
> population: uniform(1..120)
> - name: e
> size: uniform(4..4)
> population: uniform(1..100)
> - name: f
> size: fixed(4)
> population: fixed(1)
> cluster: fixed(100)
> - name: g
> size: fixed(40)
> population: fixed(1)
> cluster: fixed(10)
> - name: h
> size: fixed(4)
> population: fixed(1)
> - name: i
> size: fixed(400000)
> population: fixed(10)
> - name: j
> size: uniform(40..40)
> population: fixed(1)
> - name: k
> size: fixed(8)
> population: fixed(1)
> - name: l
> size: fixed(1)
> population: fixed(1)
> - name: m
> size: uniform(40..40)
> population: uniform(1..2000)
> - name: n
> size: gaussian(4..4)
> population: gaussian(1..100)
> - name: o
> size: fixed(4)
> population: fixed(1)
> - name: p
> size: gaussian(8..8)
> population: gaussian(1..100)
> - name: q
> size: gaussian(8..8)
> population: gaussian(1..10000)
> - name: r
> size: fixed(40)
> population: fixed(2)
> - name: s
> size: gaussian(8..8)
> population: gaussian(1..200000)
> ### Batch Ratio Distribution Specifications ###
> insert:
> partitions: fixed(1) # Our partition key is the domain so only
> insert one per batch
> select: fixed(1)/1000 # We have 1000 posts per domain so 1/1000
> will allow 1 post per batch
> batchtype: UNLOGGED # Unlogged batches
> #
> # A list of queries you wish to run against the schema
> #
> queries:
> likelyquery0:
> cql: Select * from test where a = ? and c = ? and d = ? and b = ? and e =
> ?
> fields: samerow
> likelyquery1:
> cql: Select * from test where a = ? and c = ? and d = ? and b = ? and e =
> ? and m = ?
> fields: samerow
> likelyquery2:
> cql: Select * from test where a = ? and c = ? and d = ? and b = ? and e =
> ? and m = ? and f = ?
> fields: samerow
> likelyquery3:
> cql: Select * from test where a = ? and c = ? and d = ? and b = ? and e =
> ? and m = ? and f = ? and g = ?
> fields: samerow
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
