Hey Tony, the number of tests run could vary depending on how the build is configured. For example, YARN-related tests would only run when the yarn profile is turned on. Java 8 tests would only run under Java 8.
Although I don't know that there's any reason to believe the IBM JVM has a problem with Spark, I see this issue that is potentially related to endian-ness : https://issues.apache.org/jira/browse/SPARK-2018 I don't know if that was a Spark issue. Certainly, would be good for you to investigate if you are interested in resolving it. The Jenkins output shows you exactly what tests were run and how -- have a look at the logs. https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/consoleFull On Fri, Jan 9, 2015 at 9:15 AM, Tony Reix <tony.r...@bull.net> wrote: > Hi Ted > > Thanks for the info. > However, I'm still unable to understand how the page: > > https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/ > has been built. > This page contains details I do not find in the page you indicated to me: > > https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/consoleFull > > As an example, I'm still unable to find these details: > org.apache.spark<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark/> > 12 mn 0 > 1 > 247 > 248 > > org.apache.spark.api.python<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.api.python/> > 20 ms 0 > 0 > 2 > 2 > > org.apache.spark.bagel<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.bagel/> > 7.7 s 0 > 0 > 4 > 4 > > org.apache.spark.broadcast<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.broadcast/> > 43 s 0 > 0 > 17 > 17 > > org.apache.spark.deploy<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.deploy/> > 16 s 0 > 0 > 29 > 29 > > org.apache.spark.deploy.worker<https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/org.apache.spark.deploy.worker/> > 0.55 s 0 > 0 > 12 > 12 > > ........ > > > Moreover, in my Ubuntu/x86_64 environment, I do not find 3745 tests and 0 > failures, but 3485 tests and 4 failures (when using Oracle JVM 1.7 ). When > using IBM JVM, there are only 2566 tests and 5 failures (in same component: > Streaming). > > On my PPC64BE (BE = Big-Endian)environment, the tests block after 2 hundreds > of tests. > Is Spark independent of Little/Big-Endian stuff ? > > On my PPC64LE (LE = Little-Endian) environment, I have 3485 tests only (like > on Ubuntu/x86_64 with IBM JVM), with 6 or 285 failures... > > So, I need to learn more about how your Jenkins environment extracts details > about the results. > Moreover, which JVM is used ? > > Do you plan to use IBM JVM in order to check that Spark and IBM JVM are > compatible ? (they already do not look to be compatible 100% ...). > > Thanks > > Tony > > IBM Coop Architect & Technical Leader > Office : +33 (0) 4 76 29 72 67 > 1 rue de Provence - 38432 Échirolles - France > www.atos.net<http://www.atos.net/> > ________________________________ > De : Ted Yu [yuzhih...@gmail.com] > Envoyé : jeudi 8 janvier 2015 17:43 > À : Tony Reix > Cc : dev@spark.apache.org > Objet : Re: Results of tests > > Here it is: > > [centos] $ > /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.0.5/bin/mvn > -DHADOOP_PROFILE=hadoop-2.4 -Dlabel=centos -DskipTests -Phadoop-2.4 -Pyarn > -Phive clean package > > > You can find the above in > https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/consoleFull > > > Cheers > > On Thu, Jan 8, 2015 at 8:05 AM, Tony Reix > <tony.r...@bull.net<mailto:tony.r...@bull.net>> wrote: > Thanks ! > > I've been able to see that there are 3745 tests for version 1.2.0 with > profile Hadoop 2.4 . > However, on my side, the maximum tests I've seen are 3485... About 300 tests > are missing on my side. > Which Maven option has been used for producing the report file used for > building the page: > > https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-1.2-Maven-with-YARN/lastSuccessfulBuild/HADOOP_PROFILE=hadoop-2.4,label=centos/testReport/ > ? (I'm not authorized to look at the "configuration" part) > > Thx ! > > Tony > > ________________________________ > De : Ted Yu [yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>] > Envoyé : jeudi 8 janvier 2015 16:11 > À : Tony Reix > Cc : dev@spark.apache.org<mailto:dev@spark.apache.org> > Objet : Re: Results of tests > > Please take a look at https://amplab.cs.berkeley.edu/jenkins/view/Spark/ > > On Thu, Jan 8, 2015 at 5:40 AM, Tony Reix > <tony.r...@bull.net<mailto:tony.r...@bull.net>> wrote: > Hi, > I'm checking that Spark works fine on a new environment (PPC64 hardware). > I've found some issues, with versions 1.1.0, 1.1.1, and 1.2.0, even when > running on Ubuntu on x86_64 with Oracle JVM. I'd like to know where I can > find the results of the tests of Spark, for each version and for the > different versions, in order to have a reference to compare my results with. > I cannot find them on Spark web-site. > Thx > Tony > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
