GitHub user ejwhite922 opened a pull request:
https://github.com/apache/incubator-rya/pull/181
Rya-307 MongoDB Rya DAO Batch Writer
## Description
Improved Rya MongoDB ingest of statements through the Sail Layer and Rya
DAO by queueing up multiple inserts at a time so can be written as a single
batch. If no statements in the batch have been written after a set time limit
then they are flushed out into the datastore. The size of the batch and the
time limit are configurable.
### Tests
Unit Tests/Integration Tests
### Links
[Jira](https://issues.apache.org/jira/browse/RYA-307)
### Checklist
- [ ] Code Review
- [ ] Squash Commits
#### People To Review
@amihalik
@meiercaleb
@DLotts
@jessehatfield
@isper3at
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ejwhite922/incubator-rya RYA-307_MongoIngest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/181.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #181
----
commit 08596dd004c5f35867c7a6780a800d2387e05676
Author: eric.white <[email protected]>
Date: 2017-07-19T13:08:32Z
RYA-307 Improved Rya MongoDB ingest of statements through the Sail Layer
and Rya DAO by queueing up multiple inserts at a time so can be written as a
single batch. If no statements in the batch have been written after a set time
limit then they are flushed out into the datastore. The size of the batch and
the time limit are configurable.
commit ec0ccc4ce1cc329edd02cfc5d09d543447ca59af
Author: eric.white <[email protected]>
Date: 2017-07-20T20:22:47Z
Rya-307 Commit #2. Added config options for flushing MongoDB batch writer.
commit 154e589a082b91c4556079be4fabd55d37480360
Author: eric.white <[email protected]>
Date: 2017-07-21T21:15:19Z
RYA_307 Commit #3. Fixed integration tests. Made BatchWriter compatible
with MongoCollection.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
