Re: Repair Management

[Blake Eggleston]

Cool. Just want to point out that if you're going to expose a command to 
terminate a repair, it should also stop any related validations a sync tasks 
that are in progress.


On May 18, 2017 at 6:02:16 PM, Cameron Zemek (came...@instaclustr.com) wrote:

Here is what I have done so far: 
https://github.com/apache/cassandra/compare/trunk...instaclustr:repair_management
 

> I'm not sure what you mean by "coordinator repair commands". Do you mean 
full repairs? 

By coordinator repair I meant the repair command from the coordinator node. 
That is the repair command from StorageService::repairAsync . Hopefully the 
branch above shows what I am mean. 





On 19 May 2017 at 03:16, Blake Eggleston  wrote: 

> I am looking to improve monitoring and management of repairs (so far I 
> have 
> patch for adding ActiveRepairs to table/keyspace metrics) and come across 
> ActiveRepairServiceMBean but this appears to be limited to incremental 
> repairs. Is there a reason for this 
> 
> The incremental repair stuff was just the first set of jmx controls added 
> to ActiveRepairService. ActiveRepairService is involved in all repairs 
> though. 
> 
> I was looking to add something very similar to this nodetool repair_admin 
> but it would work on co-ordinator repair commands. 
> 
> 
> I'm not sure what you mean by "coordinator repair commands". Do you mean 
> full repairs? 
> 
> What is the purpose of the current repair_admin? If I wish to add the 
> above 
> should I rename the MBean to say 
> org.apache.cassandra.db:type=IncrementalRepairService and the nodetool 
> command to inc_repair_admin ? 
> 
> 
> nodetool help repair_admin says it's purpose is to "list and fail 
> incremental repair sessions". However, by failing incremental repair 
> sessions, it doesn't mean that it cancels the validation/sync, just that it 
> releases the sstables that were involved in the repair back into the 
> unrepaired data set. I don't see any reason why you couldn't add this 
> functionality to the existing RepairService mbean. That said, before 
> getting into mbean names, it's probably best to come up with a plan for 
> cancelling validation and sync on each of the replicas involved in a given 
> repair. As far as I know (though I may be wrong), that's not currently 
> supported. 
> 
> On May 17, 2017 at 7:36:51 PM, Cameron Zemek (came...@instaclustr.com) 
> wrote: 
> 
> I am looking to improve monitoring and management of repairs (so far I 
> have 
> patch for adding ActiveRepairs to table/keyspace metrics) and come across 
> ActiveRepairServiceMBean but this appears to be limited to incremental 
> repairs. Is there a reason for this? 
> 
> I was looking to add something very similar to this nodetool repair_admin 
> but it would work on co-ordinator repair commands. 
> 
> For example: 
> $ nodetool repair_admin --list 
> Repair#1 mykeyspace columnFamilies=colfamilya,colfamilyb; 
> incremental=True; 
> parallelism=parallel progress=5% 
> 
> $ nodetool repair_admin --terminate 1 
> Terminating repair command #1 (19f00c30-1390-11e7-bb50-ffb920a6d70f) 
> 
> $ nodetool repair_admin --terminate-all # calls 
> ssProxy.forceTerminateAllRepairSessions() 
> Terminating all repair sessions 
> Terminated repair command #2 (64c44230-21aa-11e7-9ede-cd6eb64e3786) 
> 
> What is the purpose of the current repair_admin? If I wish to add the 
> above 
> should I rename the MBean to say 
> org.apache.cassandra.db:type=IncrementalRepairService and the nodetool 
> command to inc_repair_admin ? 
> 
> 

Re: Integrating vendor-specific code and developing plugins

[大平怜]

Hi all,

Everybody seems to agree with improving the plugin ecosystem (as well
as not small amount of effort needed to do that), but about
vendor-specific code integration, let me summarize the issues raised
so far.

1) How to test it?  What if my code breaks the vendor-specific build?
2) How to maintain it?  Who is to maintain the code?
3) How does it affect the Cassandra release cycle?
4) How to remove it?  It might be hard to remove once integrated, from
both technical and markting perspective.

I think #3 and #4 are rather general issues for any newly proposed
changes, while #1 and #2 are the most problematic for niche :-)
platform specific code.  #1 is technically solvable, for example, as
Jeff (thanks!) showed with the Jenkins slave at ASF and as we are
trying to connect a ppc machine with a CAPI device to the CI.

#2 must be socially solved, as a component/platform maintainer system
should be introduced like some other Apache projects.  Is there any
chance to have such a system in Cassandra?


Thanks,
Rei Odaira

2017-05-18 12:36 GMT-05:00 Jeff Jirsa :
>
>
> On Thu, May 18, 2017 at 10:28 AM, Jeff Jirsa  wrote:
>>
>>
>>
>> On Mon, May 15, 2017 at 5:25 PM, Jeremiah D Jordan
>>  wrote:
>>>
>>>
>>>
>>> To me testable means that we can run the tests at the very least for
>>> every release, but ideally they would be run more often than that.
>>> Especially with the push to not release unless the test board is all
>>> passing, we should not be releasing features that we don’t have a test board
>>> for.  Ideally that means we have it in ASF CI.  If there is someone that can
>>> commit to posting results of runs from an outside CI somewhere, then I think
>>> that could work as well, but that gets pretty cumbersome if we have to check
>>> 10 different CI dashboards at different locations before every release.
>>
>>
>>
>> It turns out there's a ppc64le jenkins slave @ asf, so I've setup
>> https://builds.apache.org/view/A-D/view/Cassandra/job/cassandra-devbranch-ppc64le-testall/
>> for testing.
>>
>> Like our other devbranch-testall builds, it takes a repo+branch as
>> parameters, and runs unit tests. While the unit tests aren't passing, this
>> platform should now be considered testable.
>>
>
> (Platform != device, though, the CAPI device obviously isn't there, so the
> row cache implementation still doesn't have public testing)
>
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

SSTablesPerReadHistogram, Tracing and SSTableReader::readCount values seems incorrect

[Benjamin Lerer]

Hi,

While working on CASSANDRA-13120,
 I came to realize
that since 3.0 we are using for SSTablesPerReadHistogram, Tracing and
SSTableReader::readCount the number of SSTables for which we did a
partition lookup and not the number of SSTables that were actually merged
to produce the response.

I also noted that SSTablesPerReadHistogram and SSTableReader::readCount are
not updated for partition range queries. It makes sense for internal range
queries but not for the user queries.

As SSTableReader::readCount seems to be used to avoid compacting cold
SSTables with SizedTierd compaction, having an invalid readCound will
probably result in un-optimal compactions.

In the same way, in SinglePartitionReadCommand (https://github.com/apache/
cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/db/
SinglePartitionReadCommand.java#L816) the number used to trigger compaction
is the number of SSTables for which we did a partition lookup and not the
one of the merged SSTables.

As, partition lookups are not cheap (especially index lookups) , Jake
suggested, in an offline discussion, to keep the
SSTablesPerReadHistogram metric
as it is and to add a new one mergedSSTablePerReadHistogram to track how
many SSTables have been actually merged.

The number of actually merged SSTables should also be the one used for the
Trace message and for determining if the SSTables must be compacted.
Before doing these changes I wanted to check if one of you had some concern
with them as I might have underestimated their impact.

Benjamin

Reg:- Data Modelling Documentation

[@Nandan@]

Hi Team,
Just as Information, When Data modeling Document will be published on
official link.
Waiting for its so long time.

Please update the document. Currently no any documents present. And please
put document, which can be understood by the absolute beginner as well as
others also.

Thanks in Advance.
Best Regards,
Nandan Priyadarshi