[jira] [Updated] (OAK-1994) Oak Console improvements
[ https://issues.apache.org/jira/browse/OAK-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-1994: - Summary: Oak Console improvements (was: Oak Console improvemenst) Oak Console improvements Key: OAK-1994 URL: https://issues.apache.org/jira/browse/OAK-1994 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Labels: console Fix For: 1.1 Oak console needs minor improvements. This bug is meant to cappture task related to such improvements -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (OAK-1994) Oak Console improvemenst
Chetan Mehrotra created OAK-1994: Summary: Oak Console improvemenst Key: OAK-1994 URL: https://issues.apache.org/jira/browse/OAK-1994 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Fix For: 1.1 Oak console needs minor improvements. This bug is meant to cappture task related to such improvements -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (OAK-1994) Limit no of children listed with ls command in Oak Console
[ https://issues.apache.org/jira/browse/OAK-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-1994: - Summary: Limit no of children listed with ls command in Oak Console (was: Oak Console improvements) Limit no of children listed with ls command in Oak Console -- Key: OAK-1994 URL: https://issues.apache.org/jira/browse/OAK-1994 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Labels: console Fix For: 1.1 Oak console needs minor improvements. This bug is meant to cappture task related to such improvements -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (OAK-1994) Limit no of children listed with ls command in Oak Console
[ https://issues.apache.org/jira/browse/OAK-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra resolved OAK-1994. -- Resolution: Fixed Fixed with http://svn.apache.org/r1613354 bq. ls no of nodes One can provide an optional param of no of nodes to be listed. Also fixed an issue with cd command to change to absolute paths Limit no of children listed with ls command in Oak Console -- Key: OAK-1994 URL: https://issues.apache.org/jira/browse/OAK-1994 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Labels: console Fix For: 1.1 Oak Console 'ls' command currently lists down all the child node which cause issue for node have large no of children. As a fix ls command should dump max say 50 child node and allow user to change the limit as part of arguments -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (OAK-1994) Limit no of children listed with ls command in Oak Console
[ https://issues.apache.org/jira/browse/OAK-1994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chetan Mehrotra updated OAK-1994: - Description: Oak Console 'ls' command currently lists down all the child node which cause issue for node have large no of children. As a fix ls command should dump max say 50 child node and allow user to change the limit as part of arguments (was: Oak console needs minor improvements. This bug is meant to cappture task related to such improvements) Limit no of children listed with ls command in Oak Console -- Key: OAK-1994 URL: https://issues.apache.org/jira/browse/OAK-1994 Project: Jackrabbit Oak Issue Type: Improvement Components: run Reporter: Chetan Mehrotra Assignee: Chetan Mehrotra Priority: Minor Labels: console Fix For: 1.1 Oak Console 'ls' command currently lists down all the child node which cause issue for node have large no of children. As a fix ls command should dump max say 50 child node and allow user to change the limit as part of arguments -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (OAK-1990) Utility js methods to manage Oak data in Mongo
[
https://issues.apache.org/jira/browse/OAK-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-1990:
-
Attachment: (was: oak-mongo.js)
Utility js methods to manage Oak data in Mongo
--
Key: OAK-1990
URL: https://issues.apache.org/jira/browse/OAK-1990
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mongomk
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Attachments: oak-mongo.js
To simplify making sense of data created by Oak in Mongo it would be helpful
to have a collection of utility js methods. This method can then be used
within Mongo shell
As a start I have put up some methods in [1]. These can be used as shown below
{noformat}
$ wget
https://gist.githubusercontent.com/chetanmeh/836ca8fffc4c410daed2/raw/oak-mongo.js
$ mongo localhost/oak --shell oak-mongo.js
MongoDB shell version: 2.6.3
connecting to: localhost/oak
type help for help
oak.countChildren('/oak:index/')
356787
oak.getChildStats('/oak:index')
{ count : 356788, size : 127743372, simple : 121.83 MB }
oak.getChildStats('/')
{ count : 593191, size : 302005011, simple : 288.01 MB }
{noformat}
[1] https://gist.github.com/chetanmeh/836ca8fffc4c410daed2
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (OAK-1990) Utility js methods to manage Oak data in Mongo
[
https://issues.apache.org/jira/browse/OAK-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chetan Mehrotra updated OAK-1990:
-
Attachment: oak-mongo.js
Utility js methods to manage Oak data in Mongo
--
Key: OAK-1990
URL: https://issues.apache.org/jira/browse/OAK-1990
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mongomk
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Attachments: oak-mongo.js
To simplify making sense of data created by Oak in Mongo it would be helpful
to have a collection of utility js methods. This method can then be used
within Mongo shell
As a start I have put up some methods in [1]. These can be used as shown below
{noformat}
$ wget
https://gist.githubusercontent.com/chetanmeh/836ca8fffc4c410daed2/raw/oak-mongo.js
$ mongo localhost/oak --shell oak-mongo.js
MongoDB shell version: 2.6.3
connecting to: localhost/oak
type help for help
oak.countChildren('/oak:index/')
356787
oak.getChildStats('/oak:index')
{ count : 356788, size : 127743372, simple : 121.83 MB }
oak.getChildStats('/')
{ count : 593191, size : 302005011, simple : 288.01 MB }
{noformat}
[1] https://gist.github.com/chetanmeh/836ca8fffc4c410daed2
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (OAK-1892) OrderedIndexConcurrentClusterIT takes too long
[ https://issues.apache.org/jira/browse/OAK-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074206#comment-14074206 ] Davide Giannella commented on OAK-1892: --- Re-enabled the test in http://svn.apache.org/r1613050 and so far the build is not timing out. [~mreutegg] can we consider this resolved? OrderedIndexConcurrentClusterIT takes too long -- Key: OAK-1892 URL: https://issues.apache.org/jira/browse/OAK-1892 Project: Jackrabbit Oak Issue Type: Bug Components: jcr Environment: trunk and 1.0 branch Reporter: Marcel Reutegger Assignee: Davide Giannella Fix For: 1.1 Attachments: OAK-1892 - Inserts.pdf, benchmarks - OAK-1892.csv The OrderedIndexConcurrentClusterIT takes too long and times out on travis. See e.g. https://travis-ci.org/apache/jackrabbit-oak/builds/27445383 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (OAK-1855) Travis builds time out
[ https://issues.apache.org/jira/browse/OAK-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074209#comment-14074209 ] Davide Giannella commented on OAK-1855: --- I'm not that expert in how to configure those stuff but I think we can address this in two steps: First split the build with multiple profiles to avoid it failing. I would say to have a profile for unit test (mvn clean install), one for pedantic (-P pedantic) and one for the integration testing (-P integrationTesting). They should all be there. Is it possible? The we can investigate the build itself drawing some stats from previous build and address what can be addressed with individual issues grouped by an epic. Thoughts? Travis builds time out -- Key: OAK-1855 URL: https://issues.apache.org/jira/browse/OAK-1855 Project: Jackrabbit Oak Issue Type: Bug Reporter: Marcel Reutegger Recently quite many travis builds failed because they hit the 50 minutes time limit. Possible solutions that come to my mind: - reduce the time our tests take - split up the build with multiple profiles -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (OAK-1990) Utility js methods to manage Oak data in Mongo
[
https://issues.apache.org/jira/browse/OAK-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074267#comment-14074267
]
Chetan Mehrotra commented on OAK-1990:
--
After discussing with [~mduerig] we decided to add this to oak-run for now. For
now one get get it from svn directly (via http) later we can dump the file via
oak-run command
http://svn.apache.org/r1613379
Utility js methods to manage Oak data in Mongo
--
Key: OAK-1990
URL: https://issues.apache.org/jira/browse/OAK-1990
Project: Jackrabbit Oak
Issue Type: New Feature
Components: mongomk
Reporter: Chetan Mehrotra
Assignee: Chetan Mehrotra
Attachments: oak-mongo.js
To simplify making sense of data created by Oak in Mongo it would be helpful
to have a collection of utility js methods. This method can then be used
within Mongo shell
As a start I have put up some methods in [1]. These can be used as shown below
{noformat}
$ wget
https://gist.githubusercontent.com/chetanmeh/836ca8fffc4c410daed2/raw/oak-mongo.js
$ mongo localhost/oak --shell oak-mongo.js
MongoDB shell version: 2.6.3
connecting to: localhost/oak
type help for help
oak.countChildren('/oak:index/')
356787
oak.getChildStats('/oak:index')
{ count : 356788, size : 127743372, simple : 121.83 MB }
oak.getChildStats('/')
{ count : 593191, size : 302005011, simple : 288.01 MB }
{noformat}
[1] https://gist.github.com/chetanmeh/836ca8fffc4c410daed2
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (OAK-1453) MongoMK failover support for replica sets (esp. shards)
[ https://issues.apache.org/jira/browse/OAK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074477#comment-14074477 ] Chetan Mehrotra commented on OAK-1453: -- For reference from this [thread|https://groups.google.com/d/msg/mongodb-user/qSi8RmvcAUY/PjvrMGpeHDcJ] bq. Now, if you are getting back success for the write and you are using writeConcern w:1 (acknowledged) but during the data load you are losing the primary, *unless you have used w:2 as write concern (waiting for replication to at least one other node before acknowledgement) you will potentially have some records that would be written to the primary, not replicated and then if you shut down the primary and another node becomes the primary, the original node will have to roll back that write (since it's not on the new primary)*. If that's what happened, you will be able to find a rollback directory in the data directory which will have the documents that were rolled back. MongoMK failover support for replica sets (esp. shards) --- Key: OAK-1453 URL: https://issues.apache.org/jira/browse/OAK-1453 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Michael Marth Assignee: Thomas Mueller Labels: production, resilience Fix For: 1.1 With OAK-759 we have introduced replica support in MongoMK. I think we still need to address the resilience for failover from primary to secoandary: Consider a case where Oak writes to the primary. Replication to secondary is ongoing. During that period the primary goes down and the secondary becomes primary. There could be some half-replicated MVCC revisions, which need to be either discarded or be ignored after the failover. This might not be an issue if there is only one shard, as the commit root is written last (and replicated last) But with 2 shards the the replication state of these 2 shards could be inconsistent. Oak needs to handle such a situation without falling over. If we can detect a Mongo failover we could query Mongo which revisions are fully replicated to the new primary and discard the potentially half-replicated revisions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (OAK-1453) MongoMK failover support for replica sets (esp. shards)
[ https://issues.apache.org/jira/browse/OAK-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074482#comment-14074482 ] Chetan Mehrotra commented on OAK-1453: -- Couple of observation regarding importance of config servers when sharding is involved based on this [thread|https://groups.google.com/d/msg/mongodb-user/Q0yRpr-kNco/DLMtpjZq36IJ] * you cannot simply randomise the list of config servers per mongos. The --configdb string needs to be the same across all mongos'. * Traffic between mongos and config servers can be very high (particularly first config server) if balancing is going on * On AWS it might be tempting to use t1.micro for config server but that should not be done as they might require higher network throughput. Use m3.large MongoMK failover support for replica sets (esp. shards) --- Key: OAK-1453 URL: https://issues.apache.org/jira/browse/OAK-1453 Project: Jackrabbit Oak Issue Type: Improvement Components: mongomk Reporter: Michael Marth Assignee: Thomas Mueller Labels: production, resilience Fix For: 1.1 With OAK-759 we have introduced replica support in MongoMK. I think we still need to address the resilience for failover from primary to secoandary: Consider a case where Oak writes to the primary. Replication to secondary is ongoing. During that period the primary goes down and the secondary becomes primary. There could be some half-replicated MVCC revisions, which need to be either discarded or be ignored after the failover. This might not be an issue if there is only one shard, as the commit root is written last (and replicated last) But with 2 shards the the replication state of these 2 shards could be inconsistent. Oak needs to handle such a situation without falling over. If we can detect a Mongo failover we could query Mongo which revisions are fully replicated to the new primary and discard the potentially half-replicated revisions. -- This message was sent by Atlassian JIRA (v6.2#6252)
