[
https://issues.apache.org/jira/browse/USERGRID-1270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15203618#comment-15203618
]
ASF GitHub Bot commented on USERGRID-1270:
------------------------------------------
GitHub user michaelarusso opened a pull request:
https://github.com/apache/usergrid/pull/493
USERGRID-1270 - Iterate over graph edge shards using a more precise
start/end column range scan.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/michaelarusso/usergrid sharding-enhancements
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/usergrid/pull/493.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #493
----
commit 8c725f19aa30a1cca5b71017c8a43586b6e4d544
Author: Michael Russo <[email protected]>
Date: 2016-03-14T23:07:19Z
Initial work to iterate over the shards with more context so we don't fetch
all rows (shards) at the same time always.
commit 92fae0df4d99e6e5d14811f6ba3680487a0b71b9
Author: Michael Russo <[email protected]>
Date: 2016-03-15T00:27:16Z
Set verbose logging to trace.
commit 4bbebc5fd759efe59bae612c9f47e36589750982
Author: Michael Russo <[email protected]>
Date: 2016-03-16T04:59:21Z
Clean up the logging, ensure the order of shard iterator within
MultiRowColumnIterator is correct. Restore NodeShardCache logic.
commit b112488db1ef01ee8417e35e48e428c40d0aa206
Author: Michael Russo <[email protected]>
Date: 2016-03-17T00:25:22Z
Make the shard consistency tests a little smarter. Update shard compaction
to be a safer background process by updating the edge writes to be atomic, and
the deletes delayed (ensures data will always be available for seeking and we
filter any dupes).
commit 4e407ff690f220ee04c535a5ce91ca5a3a07ad1d
Author: Michael Russo <[email protected]>
Date: 2016-03-18T05:12:51Z
Add 'smart' shard seeking into the multi row column iterator. This
leverages a stored 'end' edge marked on a shard at the end of compacting and
balancing a shard's (row's) edges (columns).
commit 58ae197ea581f271de644760531a9cd45287c7c9
Author: Michael Russo <[email protected]>
Date: 2016-03-18T21:31:30Z
Throttle the compactions and auditing such that the 'type' can only be
compacted one at a time (source or target node shard(s))
commit bec5093978175c87b7d76f66c8a503f062275ead
Author: Michael Russo <[email protected]>
Date: 2016-03-21T00:49:10Z
Reset original multi-row column iterator and use a new class for the smart
iterating over edge shards. Change to system time for 'last write wins' in
cassandra instead of a shard's 'createdTime'.
----
> Enhance shard compaction to track the 'end' of a shard's valid Edges
> --------------------------------------------------------------------
>
> Key: USERGRID-1270
> URL: https://issues.apache.org/jira/browse/USERGRID-1270
> Project: Usergrid
> Issue Type: Story
> Reporter: Michael Russo
> Assignee: Michael Russo
>
> During the shard compaction process, columns (edges) are moved from a source
> shard (row) to a target shard to balance the size against the configured
> shard size in Usergrid.
> This causes tombstones to be created and can make seeks slower or potentially
> cause timeout exceptions in Astyanax if the tombstone overwhelming exception
> from Cassandra is thrown.
> Implement the ability to 'mark' a shard's last valid edge so that can be fed
> into the range scan when retrieving the row (shard).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
