https://bugzilla.wikimedia.org/show_bug.cgi?id=63371

--- Comment #4 from christ...@quelltextlich.at ---
Upstream bug seems to be
  https://issues.apache.org/jira/browse/HADOOP-8900

That's included Hadoop 1.2.0, but the Pig snapshot version we used up
to now for Wikipedia Zero is Hadoop <1.2.0.

Rebuilding the current Pig head from sources also uses Hadoop <1.2.0.

Cloudera picks up the upstream bug with CDH 4.2.0. However, the CDH
4.2.0 pig jar from

 
https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/pig/pig/0.10.0-cdh4.2.0/pig-0.10.0-cdh4.2.0.jar

does not include dependencies and fails with

  Exception in thread "main" java.lang.NoClassDefFoundError:
jline/ConsoleReaderInputStream
          at java.lang.Class.getDeclaredMethods0(Native Method)
  [...]

.

Adding all dependencies by hand would be heavy lifting.

However, Cloudera's archive at

  http://archive-primary.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz

holds the full sources after the build completed. So in that archive

  pig-0.10.0-cdh4.2.0.jar

is the jar with full dependencies that can be used to run pig in local
mode without having to extend the classpath by hand.

Using that jar, the carrier file could get generated again.

Doing some more tests tomorrow to make sure the switch in the used pig
version does not affect numbers.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l

Reply via email to