GitHub user srowen opened a pull request:
https://github.com/apache/spark/pull/1833
SPARK-2903 [BUILD] [SQL] Spark SQL tests fail to compile due to dependency
structure, misplaced test class
Since recently, I find that the SQL modules' test code fails to compile
with a simple `mvn clean compile test-compile`. `sql/core` fails with a large
number of errors, beginning like so:
```
[error]
/Users/srowen/Documents/spark/sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala:23:
not found: type PlanTest
[error] class QueryTest extends PlanTest {
[error] ^
[error]
/Users/srowen/Documents/spark/sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala:28:
package org.apache.spark.sql.test is not a value
[error] test("SPARK-1669: cacheTable should be idempotent") {
...
```
I think a couple files and dependencies are in the wrong place, and it is
resolved with the following:
- Move `org.apache.spark.sql.test.TestSQLContext` from under
`sql/core/src/main/scala`, to `sql/core/src/test/scala` , which seems like
where it belongs as test code
- Move `org.apache.spark.sql.parquet.ParquetTestData` likewise from main to
test
- QueryTest in sql/core depends on `PlanTest`, which appears to be
catalyst-specific, instead of `FunSuite`. It shouldn't, and isn't needed.
- Remove dependency from `sql/core` tests on `sql/catalyst` tests, which
seems reversed
- Introduce dependency from `sql/hive` tests on `sql/core` tests to restore
access to the files moved above
What I'm still not clear on is why it has only started failing to compile
in the last week or so. But, these seem like the right changes to make. It
compiles and tests pass.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srowen/spark SPARK-2903
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1833.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1833
----
commit 96a47268eddd5d03b22cb20af71c510b430854aa
Author: Sean Owen <[email protected]>
Date: 2014-08-07T08:36:10Z
- Move org.apache.spark.sql.test.TestSQLContext from under
sql/core/src/*main*/scala, to sql/core/src/*test*/scala , which seems like
where it belongs as test code
- Move org.apache.spark.sql.parquet.ParquetTestData likewise from main to
test
- QueryTest in sql/core depends on FunSuite instead of PlanTest, which
appears to be catalyst-specific and not needed
- Remove dependency from sql/core tests on sql/catalyst tests, which seems
reversed
- Introduce dependency from sql/hive tests on sql/core tests to restore
access to the files moved above.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
