Below is stat from my latest test run which took 83 minutes. Is there anyone
else having the same problem?
Pi
test-local:
testOneMode:
[mkdir] Created dir: /root/IdeaProjects/Pig/build/test/logs-local
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 1.523 sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.507 sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.414 sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.047 sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.486 sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.257 sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.208 sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.303 sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.201 sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 8.903 sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 2.09 sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.579 sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 3.581 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 1.677 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.553 sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 53.791 sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.17 sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 57.749 sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.428 sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.317 sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.213 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.938 sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.501 sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.283 sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 1.338 sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.477 sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.661 sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 2.196 sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 1.187 sec
test-mapreduce:
testOneMode:
[mkdir] Created dir: /root/IdeaProjects/Pig/build/test/logs-mapreduce
[junit] Running org.apache.pig.test.TestAlgebraicEval
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 258.721 sec
[junit] Running org.apache.pig.test.TestBinaryStorage
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 184.114 sec
[junit] Running org.apache.pig.test.TestBuiltin
[junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 4.538 sec
[junit] Running org.apache.pig.test.TestCmdLineParser
[junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 0.07 sec
[junit] Running org.apache.pig.test.TestCombiner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 173.987 sec
[junit] Running org.apache.pig.test.TestCompressedFiles
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 174.538 sec
[junit] Running org.apache.pig.test.TestCustomSlicer
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 236.571 sec
[junit] Running org.apache.pig.test.TestDataBag
[junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 1.478 sec
[junit] Running org.apache.pig.test.TestDataModel
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.165 sec
[junit] Running org.apache.pig.test.TestEvalPipeline
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 360.121 sec
[junit] Running org.apache.pig.test.TestExGenCogroup
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 160.593 sec
[junit] Running org.apache.pig.test.TestExGenEval
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 158.003 sec
[junit] Running org.apache.pig.test.TestFilterOpNumeric
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 262.014 sec
[junit] Running org.apache.pig.test.TestFilterOpString
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 234.839 sec
[junit] Running org.apache.pig.test.TestGrunt
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.159 sec
[junit] Running org.apache.pig.test.TestInfixArithmetic
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 53.881 sec
[junit] Running org.apache.pig.test.TestLogicalPlanBuilder
[junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 1.396 sec
[junit] Running org.apache.pig.test.TestMapReduce
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 54.656 sec
[junit] Running org.apache.pig.test.TestMapReduceResultRecycling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 198.565 sec
[junit] Running org.apache.pig.test.TestParamSubPreproc
[junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 0.305 sec
[junit] Running org.apache.pig.test.TestParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 158.771 sec
[junit] Running org.apache.pig.test.TestPigContext
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.762 sec
[junit] Running org.apache.pig.test.TestPigFile
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 3.69 sec
[junit] Running org.apache.pig.test.TestPigScriptParser
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.336 sec
[junit] Running org.apache.pig.test.TestPigServer
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 158.031 sec
[junit] Running org.apache.pig.test.TestPigSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 258.747 sec
[junit] Running org.apache.pig.test.TestReversibleLoadStore
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 160.657 sec
[junit] Running org.apache.pig.test.TestStore
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 280.73 sec
[junit] Running org.apache.pig.test.TestStreaming
[junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 286.489 sec
test:
BUILD SUCCESSFUL
Total time: 83 minutes 8 seconds
On Tue, May 13, 2008 at 12:40 AM, pi song <[EMAIL PROTECTED]> wrote:
> When you run your tests next time, could you please zip all logs and send
> to me?
> This has been really annoying. I'll have to fix it.
>
> Pi
>
>
> On Tue, May 13, 2008 at 12:36 AM, Olga Natkovich <[EMAIL PROTECTED]>
> wrote:
>
> > I also see a lot of "noise" in my logs. I don't remember seeing this
> > before though it did not see to effect speed of my tests. I wonder if
> > this is somehow related to the test changes that happened recently.
> >
> >
> > ________________________________
> >
> > From: pi song [mailto:[EMAIL PROTECTED]
> > Sent: Monday, May 12, 2008 7:30 AM
> > To: [email protected]
> > Subject: Re: Slow unit testing
> >
> >
> > After looking at the log files:-
> > 1) It is not between tests but within the tests. Just my screen
> > is updating too slowly.
> > 2) My log files look very messy. Is your file as messy as mine?
> >
> > Pi
> >
> >
> > On Tue, May 13, 2008 at 12:20 AM, pi song <[EMAIL PROTECTED]>
> > wrote:
> >
> >
> > This started to happen since the introduction of
> > Hadoop16 (which includes changes in HOD). The time shown in ANT-Junit is
> > not accurate. What I can see here is that time spent doing nothing
> > between each tests is long but not taken into account.
> >
> > Is it possible to be something to do with connecting to
> > HOD?
> >
> > [junit] Tests run: 8, Failures: 0, Errors: 0, Time
> > elapsed: 262.014 sec
> > [junit] Running
> > org.apache.pig.test.TestFilterOpString
> > [junit] Tests run: 6, Failures: 0, Errors: 0, Time
> > elapsed: 234.839 sec
> > [junit] Running org.apache.pig.test.TestGrunt
> > [junit] Tests run: 1, Failures: 0, Errors: 0, Time
> > elapsed: 0.159 sec
> >
> > Pi
> >
> >
> > On Tue, May 13, 2008 at 12:15 AM, pi song
> > <[EMAIL PROTECTED]> wrote:
> >
> >
> > Already more than 40 minutes.
> >
> >
> > On Tue, May 13, 2008 at 12:10 AM, Olga Natkovich
> > <[EMAIL PROTECTED]> wrote:
> >
> >
> > I run the tests on the regular basis and
> > don't see a proble. MR tests
> > take about 20 minites for me. How long
> > does it take for you?
> >
> > Olga
> >
> >
> > > -----Original Message-----
> > > From: pi song
> > [mailto:[EMAIL PROTECTED]
> > > Sent: Monday, May 12, 2008 6:57 AM
> > > To: [email protected]
> > > Subject: Slow unit testing
> > >
> > > It has been a while since we moved
> > from Hadoop15 to Hadoop16
> > > that all the MapReduce unit tests run
> > significantly slower
> > > than before (at least 2-3x).
> > > What is the reason behind this? Is
> > there a way to fix?
> > >
> > > Since everyone has to run at least a
> > full test before
> > > submitting a patch.
> > > This is not productive from
> > developer's point of view.
> > >
> > > Pi
> > >
> >
> >
> >
> >
> >
> >
>
