[
https://issues.apache.org/jira/browse/USERGRID-254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14241273#comment-14241273
]
ASF GitHub Bot commented on USERGRID-254:
-----------------------------------------
GitHub user snoopdave opened a pull request:
https://github.com/apache/incubator-usergrid/pull/120
USERGRID-254: Use event system to clean-up stale indexes
DO NOT MERGE. FOR REVIEW ONLY.
This branch implements Events
https://issues.apache.org/jira/browse/USERGRID-254
When the Core module EntityManager deletes or updates Entities in the
Collections module, the Query Index records for the old versions of that Entity
need to be deleted. To enable this, we added an eventing system to the
Collections module so that callers can hook into entity create, update and
delete events.
And we added event handlers to clean up the Query Index records for those
events.
See the new package in Collections module:
org.apache.usergrid.persistence.collection.event
And in Core:
org.apache.usergrid.corepersistence.events
One issue with this PR is the AbstractIteratingQueryIT, which now requires
at least a 5ms write-delay or one of the tests will fail and then cause
subsequent tests in other classes to fail too.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/snoopdave/incubator-usergrid two-dot-o-events
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-usergrid/pull/120.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #120
----
commit 6446cd64d086608013006c2694d0b35c61e0ebd4
Author: Shawn Feldman <[email protected]>
Date: 2014-10-07T19:25:47Z
Adding EntityDeletedImpl
commit 406f32d2ee4333bc9c7fe879b659ae1809531f73
Author: Shawn Feldman <[email protected]>
Date: 2014-10-07T19:26:46Z
Merge branch 'two-dot-o' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o-events
commit f87c33a1e0b63be24db4c95e8a10821300cf0e15
Author: Shawn Feldman <[email protected]>
Date: 2014-10-07T19:34:09Z
Merge branch 'collection_multiget' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o-events
commit abd401447756ccb08f2b961f75465b04368aa08a
Author: Shawn Feldman <[email protected]>
Date: 2014-10-07T20:10:12Z
refactor tasks
commit c0c108f78c7fcc60cd2f199843573d5841a545ee
Author: Shawn Feldman <[email protected]>
Date: 2014-10-08T15:40:21Z
Merge branch 'two-dot-o' into two-dot-o-events
commit a09bdda6859d55f14be5c283339846737bcf89b5
Author: Shawn Feldman <[email protected]>
Date: 2014-10-08T15:50:17Z
fix test
commit 9bd46b5a543c88b177f514587882c9e4836d7bb6
Author: Shawn Feldman <[email protected]>
Date: 2014-10-08T16:34:00Z
delete current version
commit 23ff82e05aa38ca56d16016d65ef8d7316618299
Author: Shawn Feldman <[email protected]>
Date: 2014-10-09T00:07:06Z
latest
commit 5a7aa5b28cdd516e516d26cd453ca2bc74ab1797
Author: Shawn Feldman <[email protected]>
Date: 2014-10-09T15:22:11Z
Adding factory classes
commit d9ace9e0916e69e6b4d9d11a9c062219a5ed73c0
Author: Shawn Feldman <[email protected]>
Date: 2014-10-10T15:49:13Z
merge from two-o
commit 38f659822989926a6f52aee2556906e433f704f5
Author: grey <[email protected]>
Date: 2014-10-16T20:14:21Z
Merge remote-tracking branch 'origin/two-dot-o' into two-dot-o-events
* origin/two-dot-o: (32 commits)
Fixes bug with lifecycle manager cache lazy loading
taking extra line out
fix bug if you have other devices
Fixes issue with setup not creating indexes correctly
notifications test fix
changing to uuid
fixing query to all
fix service
real tests for devices, BA
register devices
adding collections test
fix device removal, remove devices
adding device deletion
change queue name
fixing queue prefix name
change queuename
Fixed UUID lookup bug
Moved index create to be on new application entity manager
Removed obsolete tests.
Fixes problem with deleting connections
...
Conflicts:
stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
commit 3551f3c6331e7edafc871727d0f1bdeeb2c7cb39
Author: Dave Johnson <[email protected]>
Date: 2014-10-20T13:46:03Z
Merge branch 'two-dot-o' into two-dot-o-events
Conflicts:
stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
commit cff9ee8c537e769ef93b12a11a9342a950e96f18
Author: Dave Johnson <[email protected]>
Date: 2014-10-20T13:49:40Z
Move event handlers under corepersistence package
commit 6ca862fa56573061560d8bfdad273cf75f0bdc99
Author: Dave Johnson <[email protected]>
Date: 2014-10-20T17:10:17Z
More complete implementation of EntityDeletedImpl, see the TODO item in
there for next steps.
commit fa2662788e062003d1653e05a30872978a61cd6f
Author: grey <[email protected]>
Date: 2014-10-20T18:02:46Z
Merge branch 'two-dot-o-events' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o-events
# By Todd Nine (30) and others
# Via amuramoto (12) and others
* 'two-dot-o-events' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid: (53 commits)
More complete implementation of EntityDeletedImpl, see the TODO item in
there for next steps.
Move event handlers under corepersistence package
Fixes cassandra heap
Fixes cassandra heap
Removed tokens and using vnodes
Changed health check to 10 seconds
Fixes issue with AWS props location
change access key logic
Fixed source execution order
add graphite
Fixes raid configuration to mount ephemeral drives in autoscaling
instances
add comments for all devices query
removing all devices hack
Fix syntax for installation and configure to use smtp
Fixes graphite installation
Merge branch 'two-dot-o' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o
Fix some issues found during testing.
fixing dependency tree for httpcomponents
Added kopf and bigdesk to elasticsearch
Fixes missing env variable for graphite server
...
Conflicts:
stack/corepersistence/collection/src/main/java/org/apache/usergrid/persistence/collection/impl/EntityCollectionManagerImpl.java
commit 31e2061add13c157251b8a4d401bab6bc2652c8b
Author: grey <[email protected]>
Date: 2014-10-20T20:10:55Z
Fixed issues surrounding EntityDeletedImpl and EntityVersionDeletedImpl.
commit 4474ca91dace7c14d780986d02ea8c8ef0a90620
Author: grey <[email protected]>
Date: 2014-10-22T21:04:34Z
Fixed spelling.
Changed factory to take in collectionScope.
Fixed and added in wiring for EntityEvents.
Added list support in the module.
commit 4e2e41117190feba3029238b463025b53b024450
Author: grey <[email protected]>
Date: 2014-10-22T21:06:02Z
Merge branch 'two-dot-o' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o-events
# By Shawn Feldman (13) and others
# Via Shawn Feldman (4) and others
* 'two-dot-o' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid: (28 commits)
checking additional threads
added error message to script
Changed configuration to disable system administrator approval
fix xml for gatling
cache queue names
change method to get queue url
license headers
Fixes missing license headers
add expected count to history
add gatling script
changing con path
portal should send debug flag
fix send to all parameter
fix send to all parameter
fixing query to send to all
Fixes bucket name
change default property value to turn off graphite
updated loadtest README
updated loadtest simulations
updated load tests
...
commit daebd252b14f57c8ce091ece5b2104310c94b188
Author: Shawn Feldman <[email protected]>
Date: 2014-10-23T20:30:15Z
add queuescope factory
commit c5195a4dff1d8223b04e7d9f4ac64c3fe276917e
Author: Shawn Feldman <[email protected]>
Date: 2014-10-23T21:42:35Z
Merge branch 'two-dot-o' of
https://git-wip-us.apache.org/repos/asf/incubator-usergrid into two-dot-o-events
commit 9fc4f0afa1b6a2d9b5fa3fac6e17ae0aab74ed8b
Author: Shawn Feldman <[email protected]>
Date: 2014-10-23T22:32:58Z
wire up deleted task
commit 5f28208f5fc018646fd4376216c172c792fc6d6d
Author: Dave Johnson <[email protected]>
Date: 2014-10-24T14:47:55Z
Implemented deleteEntity() from all index scopes methods, using ES delete
by query, next up: tests.
commit 6e5b185e9d1760b474f4cf8d746ebec5a47a5768
Author: grey <[email protected]>
Date: 2014-10-24T16:23:06Z
Wired in EntityDeletedFactory and changed implementations to support using
the factory.
commit 300353f02afc9156525120fcffa38a886136ec40
Author: Dave Johnson <[email protected]>
Date: 2014-10-24T21:22:19Z
Fix two of the EntityVersionCleanupTest tests, there are still 4
error/failures in that test.
commit 63bcf8de0a9e149d060fb6cafa42659acca83e78
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T14:21:13Z
Fix four tests, ignore two that appear to be works in progress.
commit 57c28ba3342d53537a883fbab1851fd8d612059a
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T14:29:19Z
Clean up no longer used classes and tests.
commit 05429c518cfbdf79f1ae4579909a882fdc9c50a2
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T14:31:24Z
Minor formatting changes only.
commit 91314e2a00635a7b3b063a01dc79eee227226b2b
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T14:35:07Z
Import cleanup only.
commit 50728fee145018e79d00c345c18c0cbc1f0e50a3
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T20:00:31Z
Fix Multi-bindings setup, which requires a Set and not a List for injected
multi-bindings.
commit bf347dabfff03a7f81bdb2045fd8ef61bf7a812e
Author: Dave Johnson <[email protected]>
Date: 2014-10-27T20:02:25Z
Give two Guice-created objects EntityDeleted and EntityVersionDeletedImpl,
access to the Spring-created EntityManagerFactory, also: test fixes.
----
> Add Event system to Collections module, use it to clean up old indices
> ----------------------------------------------------------------------
>
> Key: USERGRID-254
> URL: https://issues.apache.org/jira/browse/USERGRID-254
> Project: Usergrid
> Issue Type: Story
> Reporter: David Johnson
>
> When the Core module EntityManager deletes or updates Entities in the
> Collections module, the Query Index records for the old versions of that
> Entity need to be deleted.
> To enable this, we added an eventing system to the Collections module so that
> callers can hook into entity create, update and delete events.
> And we added event handlers to clean up the Query Index records for those
> events.
> See the new package in Collections module:
> org.apache.usergrid.persistence.collection.event
> And in Core:
> org.apache.usergrid.corepersistence.events
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
