[
https://issues.apache.org/jira/browse/SQOOP-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15142100#comment-15142100
]
Jarek Jarcec Cecho commented on SQOOP-2832:
-------------------------------------------
Here is my "poor's man solution" stab at the problem. It's well described in
the patch, but for convenience:
{code}
* Sqoop is running as much tests as possible inside one suite to safe time
starting
* miniclusters which is time consuming exercise (~40 seconds per single test
class).
* That however means that we have one output log file that recently grown to
more then
* 1GB in side and hence the usability has decreased.
*
* This listener will intercept each test and will reconfigure log4j to log
directly
* into files rather then to console (that would be forwarded by maven surefire
plugin to
* the normal log file). Each test will get it's own file which is easier for
human to
* read.
*
* We're using a counter to order log files per execution order rather then per
name as
* we can't guarantee log isolation entirely (e.g. some information relevant to
test N
* can be in log file for test N-1). It's easier to open previous log if you
immediately
* know what is the previous log.
{code}
I'm open to any other proposals - I just didn't find a different solution other
then run one suite per test class which seems undesirable. I've put the new
listener only to one suite for the time being to see how it will work on our
jenkins infrastructure first.
> Sqoop2: Precommit: Create log files for individual tests
> --------------------------------------------------------
>
> Key: SQOOP-2832
> URL: https://issues.apache.org/jira/browse/SQOOP-2832
> Project: Sqoop
> Issue Type: Bug
> Reporter: Jarek Jarcec Cecho
> Assignee: Jarek Jarcec Cecho
> Fix For: 1.99.7
>
> Attachments: SQOOP-2832.patch
>
>
> I feel that debugging anything in our pre-commit hook is currently mission
> impossible from several reasons. One of the main ones is that we're
> generating one single log file that has [grown to 1GB in
> size|https://builds.apache.org/job/PreCommit-SQOOP-Build/2185/artifact/test/target/surefire-reports/]
> with all "test related" logs and I'm not even remotely able to find anything
> there.
> In normal case we would have one log per test class, but as we've refactored
> our integration tests to run multiple classes inside single Suite, we're no
> longer have that for free. We have done this change to cut the time it takes
> to run the integration tests - before we were initializing mini clusters for
> each class, which is extra ~40 seconds per class, so I don't think that
> reverting it would be reasonable.
> We should perhaps explore other ways how to get multiple log files.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
