spark git commit: [SPARK-19695][SQL] Throw an exception if a `columnNameOfCorruptRecord` field violates requirements in json formats

2017-02-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 66c4b79af -> 769aa0f1d [SPARK-19695][SQL] Throw an exception if a `columnNameOfCorruptRecord` field violates requirements in json formats ## What changes were proposed in this pull request? This pr comes from #16928 and fixed a json

spark git commit: [SPARK-16122][CORE] Add rest api for job environment

2017-02-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/master d3147502e -> 66c4b79af [SPARK-16122][CORE] Add rest api for job environment ## What changes were proposed in this pull request? add rest api for job environment. ## How was this patch tested? existing ut. Author: uncleGen

spark git commit: [SPARK-15615][SQL] Add an API to load DataFrame from Dataset[String] storing JSON

2017-02-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master dc005ed53 -> d3147502e [SPARK-15615][SQL] Add an API to load DataFrame from Dataset[String] storing JSON ## What changes were proposed in this pull request? SPARK-15615 proposes replacing the sqlContext.read.json(rdd) with a dataset

spark git commit: [SPARK-19658][SQL] Set NumPartitions of RepartitionByExpression In Parser

2017-02-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4661d30b9 -> dc005ed53 [SPARK-19658][SQL] Set NumPartitions of RepartitionByExpression In Parser ### What changes were proposed in this pull request? Currently, if `NumPartitions` is not set in RepartitionByExpression, we will set it

spark git commit: [SPARK-19652][UI] Do auth checks for REST API access (branch-2.0).

2017-02-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.0 ddd432de2 -> 8cdd12156 [SPARK-19652][UI] Do auth checks for REST API access (branch-2.0). The REST API has a security filter that performs auth checks based on the UI root's security manager. That works fine when the UI root is the

spark git commit: [SPARK-19554][UI, YARN] Allow SHS URL to be used for tracking in YARN RM.

2017-02-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 37112fcfc -> 4661d30b9 [SPARK-19554][UI,YARN] Allow SHS URL to be used for tracking in YARN RM. Allow an application to use the History Server URL as the tracking URL in the YARN RM, so there's still a link to the web UI somewhere in YARN

spark git commit: [SPARK-19652][UI] Do auth checks for REST API access (branch-2.1).

2017-02-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.1 9a890b5fa -> 21afc4534 [SPARK-19652][UI] Do auth checks for REST API access (branch-2.1). The REST API has a security filter that performs auth checks based on the UI root's security manager. That works fine when the UI root is the

spark git commit: [SPARK-19666][SQL] Skip a property without getter in Java schema inference and allow empty bean in encoder creation

2017-02-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 1f86e795b -> 37112fcfc [SPARK-19666][SQL] Skip a property without getter in Java schema inference and allow empty bean in encoder creation ## What changes were proposed in this pull request? This PR proposes to fix two. **Skip a

spark git commit: [SPARK-19616][SPARKR] weightCol and aggregationDepth should be improved for some SparkR APIs

2017-02-22 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master e4065376d -> 1f86e795b [SPARK-19616][SPARKR] weightCol and aggregationDepth should be improved for some SparkR APIs ## What changes were proposed in this pull request? This is a follow-up PR of #16800 When doing SPARK-19456, we found

spark git commit: [SPARK-13721][SQL] Make GeneratorOuter unresolved.

2017-02-22 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master bf7bb4977 -> 10c566cc3 [SPARK-13721][SQL] Make GeneratorOuter unresolved. ## What changes were proposed in this pull request? This is a small change to make GeneratorOuter always unresolved. It is mostly no-op change but makes it more

spark git commit: [SPARK-19679][ML] Destroy broadcasted object without blocking

2017-02-22 Thread mlnick
Repository: spark Updated Branches: refs/heads/master ef3c73535 -> bf7bb4977 [SPARK-19679][ML] Destroy broadcasted object without blocking ## What changes were proposed in this pull request? Destroy broadcasted object without blocking use `find mllib -name '*.scala' | xargs -i bash -c 'egrep

spark-website git commit: Fix last update to Java pi example

2017-02-22 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site ca64fac2e -> 470b7ed51 Fix last update to Java pi example Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/470b7ed5 Tree:

spark-website git commit: Add instructions for running individual tests.

2017-02-22 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 879303593 -> ca64fac2e Add instructions for running individual tests. This is useful and I often forget how to do it. I learned some new tricks when @squito gave @jinxing64 some tips on how to do this, so I thought it was worth