GitHub user rameshch16 opened a pull request:
https://github.com/apache/spark/pull/20957
Branch 2.3
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/spark branch-2.3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20957.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20957
commit acf3b70d16cc4d2416b4ce3f42b3cf95836170ed
Author: Tathagata Das
Date: 2018-01-19T00:29:45Z
[SPARK-23142][SS][DOCS] Added docs for continuous processing
## What changes were proposed in this pull request?
Added documentation for continuous processing. Modified two locations.
- Modified the overview to have a mention of Continuous Processing.
- Added a new section on Continuous Processing at the end.
![image](https://user-images.githubusercontent.com/663212/35083551-a3dd23f6-fbd4-11e7-9e7e-90866f131ca9.png)
![image](https://user-images.githubusercontent.com/663212/35083618-d844027c-fbd4-11e7-9fde-75992cc517bd.png)
## How was this patch tested?
N/A
Author: Tathagata Das
Closes #20308 from tdas/SPARK-23142.
(cherry picked from commit 4cd2ecc0c7222fef1337e04f1948333296c3be86)
Signed-off-by: Tathagata Das
commit 225b1afdd1582cd4087e7cb98834505eaf16743e
Author: brandonJY
Date: 2018-01-19T00:57:49Z
[DOCS] change to dataset for java code in
structured-streaming-kafka-integration document
## What changes were proposed in this pull request?
In latest structured-streaming-kafka-integration document, Java code
example for Kafka integration is using `DataFrame`, shouldn't it be
changed to `DataSet`?
## How was this patch tested?
manual test has been performed to test the updated example Java code in
Spark 2.2.1 with Kafka 1.0
Author: brandonJY
Closes #20312 from brandonJY/patch-2.
(cherry picked from commit 6121e91b7f5c9513d68674e4d5edbc3a4a5fd5fd)
Signed-off-by: Sean Owen
commit 541dbc00b24f17d83ea2531970f2e9fe57fe3718
Author: Takuya UESHIN
Date: 2018-01-19T03:37:08Z
[SPARK-23054][SQL][PYSPARK][FOLLOWUP] Use sqlType casting when casting
PythonUserDefinedType to String.
## What changes were proposed in this pull request?
This is a follow-up of #20246.
If a UDT in Python doesn't have its corresponding Scala UDT, cast to string
will be the raw string of the internal value, e.g.
`"org.apache.spark.sql.catalyst.expressions.UnsafeArrayData"` if the
internal type is `ArrayType`.
This pr fixes it by using its `sqlType` casting.
## How was this patch tested?
Added a test and existing tests.
Author: Takuya UESHIN
Closes #20306 from ueshin/issues/SPARK-23054/fup1.
(cherry picked from commit 568055da93049c207bb830f244ff9b60c638837c)
Signed-off-by: Wenchen Fan
commit 54c1fae12df654c7713ac5e7eb4da7bb2f785401
Author: Sameer Agarwal
Date: 2018-01-19T09:38:08Z
[BUILD][MINOR] Fix java style check issues
## What changes were proposed in this pull request?
This patch fixes a few recently introduced java style check errors in
master and release branch.
As an aside, given that [java linting currently
fails](https://github.com/apache/spark/pull/10763
) on machines with a clean maven cache, it'd be great to find another
workaround to [re-enable the java style
checks](https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L577)
as part of Spark PRB.
/cc zsxwing JoshRosen srowen for any suggestions
## How was this patch tested?
Manual Check
Author: Sameer Agarwal
Closes #20323 from sameeragarwal/java.
(cherry picked from commit 9c4b99861cda3f9ec44ca8c1adc81a293508190c)
Signed-off-by: Sameer Agarwal
commit e58223171ecae6450482aadf4e7994c3b8d8a58d
Author: Nick Pentreath
Date: 2018-01-19T10:43:23Z
[SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter
Update user