GitHub user bigdatatraining opened a pull request:
https://github.com/apache/spark/pull/15020
Spark 2.0 error in Intellij
If i run twitter code in Console it's working fine, but if i run same
command in Spark 2.0 in Intellij I got this error
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/Logging
Not only this problem most of the programs getting same errors please let
me know why?
import org.apache.spark.Logging
Its not available in spark 2.0 How to resolve this issues?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/spark branch-2.0
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15020.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15020
commit fb944a1e85a4d0e618cf7485afb0d0b39367fbda
Author: Tom Graves
Date: 2016-07-22T11:41:38Z
[SPARK-16650] Improve documentation of spark.task.maxFailures
Clarify documentation on spark.task.maxFailures
No tests run as its documentation
Author: Tom Graves
Closes #14287 from tgravescs/SPARK-16650.
(cherry picked from commit 6c56fff118ff2380c661456755db17976040de66)
Signed-off-by: Sean Owen
commit 28bb2b0447e9b47c4c568de983adde4a49b29263
Author: Dongjoon Hyun
Date: 2016-07-22T12:20:06Z
[SPARK-16651][PYSPARK][DOC] Make `withColumnRenamed/drop` description more
consistent with Scala API
## What changes were proposed in this pull request?
`withColumnRenamed` and `drop` is a no-op if the given column name does not
exists. Python documentation also describe that, but this PR adds more explicit
line consistently with Scala to reduce the ambiguity.
## How was this patch tested?
It's about docs.
Author: Dongjoon Hyun
Closes #14288 from dongjoon-hyun/SPARK-16651.
(cherry picked from commit 47f5b88db4d65f1870b16745d3c93d01051ba20b)
Signed-off-by: Sean Owen
commit da34e8e8faaf7239f6dfe248812c83e1b2e2c1fd
Author: Cheng Lian
Date: 2016-07-23T18:41:24Z
[SPARK-16380][EXAMPLES] Update SQL examples and programming guide for
Python language binding
This PR is based on PR #14098 authored by wangmiao1981.
## What changes were proposed in this pull request?
This PR replaces the original Python Spark SQL example file with the
following three files:
- `sql/basic.py`
Demonstrates basic Spark SQL features.
- `sql/datasource.py`
Demonstrates various Spark SQL data sources.
- `sql/hive.py`
Demonstrates Spark SQL Hive interaction.
This PR also removes hard-coded Python example snippets in the SQL
programming guide by extracting snippets from the above files using the
`include_example` Liquid template tag.
## How was this patch tested?
Manually tested.
Author: wm...@hotmail.com
Author: Cheng Lian
Closes #14317 from liancheng/py-examples-update.
(cherry picked from commit 53b2456d1de38b9d4f18509e7b36eb3fbe09e050)
Signed-off-by: Reynold Xin
commit 31c3bcb46cb56b57d3cdcb8c42e7056dab0f7601
Author: Wenchen Fan
Date: 2016-07-23T18:39:48Z
[SPARK-16690][TEST] rename SQLTestUtils.withTempTable to withTempView
after https://github.com/apache/spark/pull/12945, we renamed the
`registerTempTable` to `createTempView`, as we do create a view actually. This
PR renames `SQLTestUtils.withTempTable` to reflect this change.
N/A
Author: Wenchen Fan
Closes #14318 from cloud-fan/minor4.
(cherry picked from commit 86c275206605c44e1ebca2f166d62868e44bf029)
Signed-off-by: Reynold Xin
commit 198b0426e07f3d4b1fbbef21d39daa32a75da36c
Author: Liwei Lin
Date: 2016-07-24T07:35:57Z
[SPARK-16515][SQL][FOLLOW-UP] Fix test `script` on OS X/Windows...
The current `sed` in `test_script.sh` is missing a `$`, leading to the
failure of `script` test on OS X:
```
== Results ==
!== Correct Answer - 2 == == Spark Answer - 2 ==
![x1_y1][x1]
![x2_y2][x2]
```
In addition, this `script` test would also fail on systems like Windows
where we couldn't be able to invoke `bash` or `echo | sed`.
This patch
- fixes `sed` in `test_script.sh`
- adds command guards so that the `script` test would pass on systems like
Windows
- Jenkins
- Manually verified tests pass on OS X
Author: Liwei Lin
Closes #14280 from lw-lin/osx-sed.
(cherry picked from commit d6795c7a254b83d4ae4785f3add74981e5273c91)
Signed-off-by: Sean Owen
commit d226dce12babcd9f30db033417b2b9ce79f44312
Author: Qifan Pu
Date: 2016-07-25T04:53:21Z
[SPARK-16699][S
