[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14905987#comment-14905987
]
Stefan Egli commented on OAK-3251:
--
bq. possible improvements in DocumentDiscoveryLiteServiceTest
I've created OAK-3444 to follow-up. Would take an overhaul of the whole test if
going virtual clock is tackled. I don't think it should be moved to ITs as it
then gets tested much less frequently probably.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906365#comment-14906365
]
Davide Giannella commented on OAK-3251:
---
I think then we can consider this issue as resolved. We did what was a quick
win to speed up the things and the build moved 3 minutes faster in my local.
This can work as well as analysis for identifying the slowest tests.
If no one will object I will mark it resolved on Monday.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906643#comment-14906643
]
angela commented on OAK-3251:
-
[~tripod] created that test. so, from the top of my head i probably know less
that you, now that you already managed to reduce it a bit as you state above :-)
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14905217#comment-14905217
]
Tommaso Teofili commented on OAK-3251:
--
bq. would that be fine for you if we move QueryJcrTest of oak-solr-core as IT?
nope, sorry. That's part of the unit tests we need to always run to make sure
that the Solr indexing and query works as expected from JCR.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, build-1443006309.log,
> oak-build-for-unittests-times.log, oak-build-times.log, times-1441637422.log,
> times-1443006309.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14734312#comment-14734312
]
Marcel Reutegger commented on OAK-3251:
---
The upgrade related tests are slow because sessions are still open when the
repository is shut down. This adds a 5 seconds delay per test. I'll take care
of it in OAK-3356.
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> build-1441637422.log, oak-build-for-unittests-times.log, oak-build-times.log,
> times-1441637422.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730655#comment-14730655
]
Chetan Mehrotra commented on OAK-3251:
--
bq. org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
Should be fine. Most of the logic has direct testcase support in other test in
oak-lucene
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> oak-build-for-unittests-times.log, oak-build-times.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14730517#comment-14730517
]
Davide Giannella commented on OAK-3251:
---
[~chetanm] oak-lucene/org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
covers regressions WRT JR2. Would that be fine for you moving it as
IT? It takes a considerable amount of time (131.114 sec) and to me it
is more an IT.
Does anyone have more insights about
- org.apache.jackrabbit.oak.upgrade.CopyVersionHistoryTest
- org.apache.jackrabbit.oak.upgrade.CopyVersionHistorySidegradeTest
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: build-1441353866-times.log, build-1441353866.log,
> oak-build-for-unittests-times.log, oak-build-times.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14728926#comment-14728926
]
Alex Parvulescu commented on OAK-3251:
--
+1 looks good
> speeding up the build time
> --
>
> Key: OAK-3251
> URL: https://issues.apache.org/jira/browse/OAK-3251
> Project: Jackrabbit Oak
> Issue Type: Improvement
>Reporter: Davide Giannella
>Assignee: Davide Giannella
> Attachments: oak-build-for-unittests-times.log, oak-build-times.log
>
>
> Running the build with a {mvn clean install} takes a considerable amount of
> time: 15 minutes on my local.
> The top 10 slowest unit test are the following
> {noformat}
> 110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 54.012 sec
> org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
> 36.593 sec
> org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
> 35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
> 25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
> 24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
> 17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
> 16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
> 14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
> 11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
> {noformat}
> Is there anything we could do to speed-up these?
> sorted times obtained with
> https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710742#comment-14710742
]
Marcel Reutegger commented on OAK-3251:
---
VersionGarbageCollectorTest is now an IT. See OAK-3277.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704453#comment-14704453
]
Davide Giannella commented on OAK-3251:
---
If I remember correctly it was agreed to run by default only Segment
as it's faster and it doesn't need any extra tools (Mongo, RDBs) up
and running.
Indeed the normal behaviour of a Mongo IT test is to skip if no mongo
instance is running.
The idea is that most of the functionality should be persistence
agnostic and that before committing you should ensure to give a run on
all the persistence or in case our CI should catch it.
The concept of the fixtures/profiles was initially introduced to
address Travis that was constantly timing out. By splitting in
profiles we managed to have it running. It also comes handy now on
jenkins as it should allow it to run in parallel (assuming enough
executors available).
Giving the right profile/fixture to the tests allows us to have faster
feedbacks. For example, let's say you're working on a RDB
functionality and you want to quickly check for regression you could
give a command like {{mvn clean install
-Dnsfixtures=DOCUMENT_RDB}}. This will give you a faster feedback as
it won't run any Segment or Mongo related stuff. The same for other
profiles.
Before committing, or in case we have CI, you can run {{mvn clean
install -Dnsfixture=}} and waiting your 15, 20 minutes for having a
complete check. Add integrationTesting for increasing coverage and
time ;).
{quote}
do we have documentation about how to do that properly?
{quote}
I should write it down. It takes some time and on my todo list.
{quote}
The approach that I adopted from other tests doesn't scale well at all
{quote}
happy to discuss it for alternatives. We can have a chat in case
before hand to speed up things.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704466#comment-14704466
]
Marcel Reutegger commented on OAK-3251:
---
I share Julian's view. A plain {{mvn clean install}} should run tests on all
supported backends. It may mean some DocumentStore related tests are not
executed because there is no MongoDB running, but most of the DocumentNodeStore
tests also run on the in-memory DocumentStore implementation. Those tests
should run in any case.
The CI is a different story and there it makes sense to have the fixtures
defined and test multiple backends in parallel.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704732#comment-14704732
]
Davide Giannella commented on OAK-3251:
---
[~alex.parvulescu] would you agree as well?
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704702#comment-14704702
]
Julian Reschke commented on OAK-3251:
-
FWIW, I just improved BasicDocumentStoreTest not to waste time, see OAK-3257.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702933#comment-14702933
]
Davide Giannella commented on OAK-3251:
---
Didn't look at the actual code yet so a full investigation and check
will be needed; but here's the plan. Can the mentioned ppl share their
opinions?
[~mduerig], [~alex.parvulescu]
{noformat}
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
{noformat}
- add the Fixture concept to the tests (if not already there)
- move them as IT (ie renaming the classes as *IT)
[~egli], [~reschke], [~mreutegg]
{noformat}
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
{noformat}
- add the Fixture concept to the tests (if not already there)
- move them as IT (ie renaming the classes as *IT)
[~tmueller]
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
{noformat}
- does it make sense to move them as ITs?
- if so will move them and see later on about what we can do to
actually speed-up the execution.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702912#comment-14702912
]
Michael Dürig commented on OAK-3251:
Some of those should probably be ITs instead of UTs. Personally I wouldn't mind
to move all of them to ITs.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702943#comment-14702943
]
Michael Dürig commented on OAK-3251:
I'd be fine with moving the segment tests to ITs.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702939#comment-14702939
]
Julian Reschke commented on OAK-3251:
-
I'm not convinced. I'd prefer that a regular unit test run on oak-core actually
covers all DocumentStore implementations.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703094#comment-14703094
]
Davide Giannella commented on OAK-3251:
---
[~reschke] are you referring to the Fixture thing or the IT move? Or both? For
the fixture we'll have the test running for all DOCUMENT fixtures; but you'll
have to provide the parameter on the command line as the default clean install
runs SEGMENT_MK.
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703106#comment-14703106
]
Julian Reschke commented on OAK-3251:
-
...the fixture. I can see why higher-level projects (say oak-jcr) do not need
to be unit-tested with each MK implementation by default, but I believe the
answer needs to be different for oak-core. A standard unit test run should
exercise at least basic functionality of all code that's in, and that implies
for instance running the same set of tests for each implementation of the
DocumentStore API.
If specific tests are too slow then yes we should check whether we can speed
them up, or make them ITs (BTW: do we have documentation about how to do that
properly? The approach that I adopted from other tests doesn't scale well at
all :-)
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-3251) speeding up the build time
[
https://issues.apache.org/jira/browse/OAK-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702675#comment-14702675
]
Davide Giannella commented on OAK-3251:
---
When running a simple clean install, only the SEGMENT_MK is passed as
fixture. A first quick win I can thing by looking at the tests
namespaces is to adopt the {{FixtureHelper}} in
- org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
- org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
by having them running only for {{DOCUMENT_*}} fixtures. This will
speed up a clean install already by roughly one minute.
On the other hand for speeding up the builds Fixtures oriented builds
on jenkins we could limit to {{SEGMENT_MK}} fixtures the tests
- org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
- org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
- org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
- org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
having therefore a build for {{DOCUMENT_*}} roughly 2 minutes faster.
Thoughts?
speeding up the build time
--
Key: OAK-3251
URL: https://issues.apache.org/jira/browse/OAK-3251
Project: Jackrabbit Oak
Issue Type: Improvement
Reporter: Davide Giannella
Assignee: Davide Giannella
Attachments: oak-build-for-unittests-times.log, oak-build-times.log
Running the build with a {mvn clean install} takes a considerable amount of
time: 15 minutes on my local.
The top 10 slowest unit test are the following
{noformat}
110.822 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
54.012 sec
org.apache.jackrabbit.oak.plugins.segment.SegmentDataStoreBlobGCTest
36.593 sec
org.apache.jackrabbit.oak.plugins.document.VersionGarbageCollectorTest
35.234 sec org.apache.jackrabbit.oak.jcr.query.QueryJcrTest
25.047 sec org.apache.jackrabbit.oak.plugins.segment.file.FileStoreTest
24.787 sec org.apache.jackrabbit.oak.plugins.document.BasicDocumentStoreTest
17.477 sec org.apache.jackrabbit.oak.plugins.segment.ExternalBlobTest
16.343 sec org.apache.jackrabbit.oak.jcr.query.QueryTest
14.519 sec org.apache.jackrabbit.oak.plugins.segment.CompactionAndCleanupTest
11.604 sec org.apache.jackrabbit.oak.run.osgi.LuceneSupportTest
{noformat}
Is there anything we could do to speed-up these?
sorted times obtained with
https://gist.github.com/davidegiannella/b1d3cbe51d1f70314500
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
