Thanks for clearing up some of these things. I've updated NUTCH-1081
with this thread. We should deal with this test soon. Also I'm
definitely up for remaining a single build (and just remove this extra
ant build)
On 11/15/2011 09:35 PM, Lewis John Mcgibbney wrote:
Hi Ferdy,
OK so the issue is TestAPI... here we go again ;0)
Typical output for the test can be seen here [1]. Just for the record,
Markus mentioned a while ago that we should have some place to view
these test output files, so at least we've now found them in the build
directory inside the Jenkins Nutch workspace.
I remember when we were fixing the tests before, the issue I opened
[2] actually highlighted that TestAPI was failing when I initially
opened the issue, however this miraculously resolved itself and after
lots and lots of testing I could not reproduce. I think we need to
look really closely at TestAPI, at what it does, and try to identify
where the bug is! It sounds like there is definitely some fishy
business going on.
Lets leave this issue open and have a look at it eh?
[1]
https://builds.apache.org/view/M-R/view/Nutch/job/Nutch-nutchgora/ws/nutchgora/build/test/TEST-org.apache.nutch.api.TestAPI.txt/*view*/
[2] https://issues.apache.org/jira/browse/NUTCH-1081
On Mon, Nov 14, 2011 at 11:28 AM, Ferdy Galema
<[email protected] <mailto:[email protected]>> wrote:
I've pinpointed the problem to a single test, namely TestAPI
(TestGoraStorage seems to work fine, I guess the failure at my
home workstation was a memory issue). This is what happends after
I do a clean checkout Nutchgora from my regular workstation.
"ant test"
TestAPI fails, see extracted lines from TEST-TestAPI.txt:
Caused by: java.io.IOException:
java.sql.SQLTransientConnectionException:
java.net.ConnectException: Connection refused
...
java.lang.IllegalStateException: Job in state DEFINE instead of
RUNNING
...
junit.framework.AssertionFailedError: expected:<[FAILED]> but
was:<[RUNNING]>
at org.apache.nutch.api.TestAPI.testJobsAPI(TestAPI.java:118)
The insane part is the fact that I had a working copy of Nutchgora
just a few moments ago, from which I reverted the local workspace
changes (that could have absolutely nothing to do with these
issues) whereafter the test failed again. I re-applied the
changes, but TestAPI just does not work anymore. This working copy
is 3 revisions behind head (that is head minus 1 commit of Markus
and 2 commits of Lewis). Then, "rm build/ runtime/ -r" followed by
"ant runtime" and "ant test" and suddenly TestAPI worked! But,
this is not reproducible because rerunning the tests just failed
it again. So I guess the test just randomly succeeds. I'm sorry if
this is becoming a really vague description, but it certainly is a
real vague problem. It's driving me nuts.
So the underlying question is, what is TestAPI
(NutchServer/NutchApp) and what purpose does it serve? I'm just
not familiar enough with these components.
By the way, could somebody explain what the difference is between
the following two build jobs?
https://builds.apache.org/job/Nutch-nutchgora-ant/
https://builds.apache.org/job/Nutch-nutchgora/
Because in Nutch-nutchgora-ant, TestAPI also fails:
[junit] Tests run: 4, Failures: 1, Errors: 0, Time elapsed: 11.081 sec
[junit] Test org.apache.nutch.api.TestAPI FAILED
In Nutch-nutchgora, it works perfectly fine.
Ferdy.
On 11/12/2011 08:57 PM, Andrzej Bialecki wrote:
On 12/11/2011 19:12, Ferdy Galema wrote:
Hi,
I cannot get a grasp on this. I cleanly checked out
Nutchgora, but the
tests just fail. It's TestAPI and TestGoraStorage. I
noticed this
problem before a while ago but then it seemed cleaning
caches and build
folders resolved it. Now it is rearing it's ugly head again.
Logs indicate that TestAPI failes because of assert
failures (RUNNING
job status vs something else) and also connection failures
to SQL.
TestGoraStorage failes with the multi process test
throwing "to many
open files" (but I can migitate the latter problem by
setting lowering
loop times).
Then when I enable building with HBase (decomment lines in
ivy) I get
compile errors in TestGoraStorage (at setDeamon call).
This has
something to do with multiple version of the HSQL libs?
I'm completely clueless about what this might be. I tried
everything,
also using different versions of JDK. Monday I might have
more info, as
I will be back on my pc at work where I should have a
working copy. Any
help in the meantime would be great.
Hmmm, I vaguely recall that either Hadoop or HBase comes with
an incompatible version of HSQL lib. This is a transitive
dependency of Gora, and when it's pulled in it conflicts with
the version we use in Nutch.
--
/Lewis/
