Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8746#discussion_r45070100
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/sources/DDLTestSuite.scala ---
@@ -113,4 +113,23 @@ class DDLTestSuite extends DataSourceTest
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-148177961
Hi @smola - Is there anything else that you need me to check for this PR?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-145291397
Thanks again @cloud-fan I have pushed a new commit with the updates!
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-145284551
@smola @cloud-fan Thanks again for the review guys. I have made the
additional changes that were noted.
---
If your project is set up for it, you can reply
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8746#discussion_r41091143
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DDLParser.scala
---
@@ -173,13 +174,15 @@ class DDLParser(parseQuery: String
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8746#discussion_r41091151
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/sources/DDLTestSuite.scala ---
@@ -113,4 +124,20 @@ class DDLTestSuite extends DataSourceTest
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-145031808
@smola Thanks for the review. I will add a test case for it and push
another commit today.
---
If your project is set up for it, you can reply to this email
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-145143378
@smola I have added a test case for this. Let me know if you see anything
else that needs to be changed or added.
---
If your project is set up for it, you can
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8746
[SPARK-7012][SQL] Add support for NOT NULL modifier for column definitions
on DDLParser
Add support for NOT NULL modifier for column definitions in DDLParser
You can merge this pull request
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8746#issuecomment-140074649
@smola I have created this PR for SPARK-7012
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user sabhyankar closed the pull request at:
https://github.com/apache/spark/pull/8243
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sabhyankar closed the pull request at:
https://github.com/apache/spark/pull/8248
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sabhyankar closed the pull request at:
https://github.com/apache/spark/pull/8241
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-139660661
closing per discussion on parent Jira
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user sabhyankar closed the pull request at:
https://github.com/apache/spark/pull/8249
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sabhyankar closed the pull request at:
https://github.com/apache/spark/pull/8247
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-135881841
Hi @mengxr The current approach of rebroadcasting on every predict
broadcasts the entire model and so I suppose the first issue you identified
also applies
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8248#issuecomment-135506704
@mengxr I am going to update PRs #8248 #8247 #8243 #8241 and #8249 after PR
#8241 is merged to trunk. This is needed because we want to use the common
trait
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-135121895
@feynmanliang - That sounds good - I can document the intent of the
Broadcastable trait in the scaladoc. The reason I made this trait
private[spark] is because we
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8241#discussion_r38011826
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala ---
@@ -21,16 +21,15 @@ import java.lang.{Iterable = JIterable
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-135046260
@jkbradley @feynmanliang I can certainly update the PR and change the
filename and trait name to be the same (Broadcastable).
I understand the concern
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8241#discussion_r37980863
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala ---
@@ -21,16 +21,15 @@ import java.lang.{Iterable = JIterable
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-135241069
Sounds good @jkbradley . I dont think there is anything pending on my side
for this PR. Let me know if you see something otherwise. I will update the
other PRs once
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-134796319
@feynmanliang @jkbradley thank you for the review guys. Traits will not
allow context bound (to ClassTag) for type parameters and so I ended up using
implicits
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-134806330
@feynmanliang without the ClassTag, the compiler complains that no ClassTag
is available for our type T.
No ClassTag available for T
[error] case
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-134826539
@jkbradley I have resolved the conflicts and updated the PR with the
changes identified. Let me know if you see any issues. Since I am not combining
PRs
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/8241#discussion_r37410650
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala ---
@@ -83,9 +86,13 @@ class NaiveBayesModel private[spark
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132631199
Hi @holdenk I have moved the common logic to a trait and am mixin it with
the model. Let me know if you see something that needs to be corrected. I am
still getting
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132805558
I forgot to mention, I am also only passing in the SparkContext now. I
should have done that to begin with :(
---
If your project is set up for it, you can reply
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132805093
Thanks @holdenk ! That makes sense. I moved the private var to the trait.
Let me know if you see something else that is out of place. I believe the
generic types I
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132362277
Thanks for pointing that out @holdenk ! I have pushed a change to the PR!
---
If your project is set up for it, you can reply to this email and have your
reply
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132374365
@holdenk Not sure if you are reviewing the other PRs, but the fix should
now be in all of them. Thx!
---
If your project is set up for it, you can reply
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/8241#issuecomment-132364417
@holdenk yep :) I am updating those as well!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37184243
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -254,6 +255,7 @@ class DenseMatrix(
* @param numRows number
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/7729#discussion_r37183646
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -254,6 +255,7 @@ class DenseMatrix(
* @param numRows number
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8243
[SPARK-10015][MLlib]: ML model broadcasts should be stored in private vars:
spark.ml tree ensembles
ML model broadcasts should be stored in private vars: spark.ml tree
ensembles
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8241
[SPARK-10017] [MLlib]: ML model broadcasts should be stored in private
vars: mllib NaiveBayes
ML model broadcasts should be stored in private vars: mllib NaiveBayes
You can merge this pull
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8247
[SPARK-10018][MLlib]: ML model broadcasts should be stored in private vars:
mllib clustering
Applies to: spark.mllib.clustering types:
GaussianMixtureModel
KMeansModel
LocalLDAModel
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8248
[SPARK-10019][MLlib]: ML model broadcasts should be stored in private vars:
mllib IDFModel
ML model broadcasts should be stored in private vars: mllib IDFModel.
Applies
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/8249
[SPARK-10020][MLlib]: ML model broadcasts should be stored in private vars:
mllib GeneralizedLinearModel
ML model broadcasts should be stored in private vars: mllib
GeneralizedLinearModel
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7729#issuecomment-131204955
I removed the tags from two of the toString method and confirmed left the
tags in place for the alternate constructors as mechcoder mentioned.
---
If your project
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7729#issuecomment-130777274
I am going to take a look at this and update the PR by tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7729#issuecomment-127825657
@MechCoder I couldnt find the untagged alternate constructor in
DenseMatrix. Can you point me to where you see it?
---
If your project is set up for it, you can
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7729#issuecomment-127455561
@MechCoder thank you for the very detailed review! I have made most of the
changes that you had pointed out except for the following:
- I left the tags
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7729#issuecomment-126948372
@MechCoder Thanks for pointing that out. I agree that the tag might not be
necessary for the sealed trait. Let me know if you come across anything else.
---
If your
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7740#issuecomment-126784214
Just a quick check to see if there is anything else I need to do or is
pending for this Jira? @tdas
---
If your project is set up for it, you can reply
Github user sabhyankar commented on a diff in the pull request:
https://github.com/apache/spark/pull/7740#discussion_r36008152
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -527,7 +527,9 @@ private[spark] object SparkConf extends Logging
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7740#issuecomment-125886368
@tdas @CodingCat I have put the method implementation inside braces {}
with the additional indentation for the 2nd line as needed.
@srowen I have added
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/7740
[SPARK-9056] [Streaming] Rename configuration
`spark.streaming.minRememberDuration` to
`spark.streaming.fileStream.minRememberDuration`
Rename configuration `spark.streaming.minRememberDuration
Github user sabhyankar commented on the pull request:
https://github.com/apache/spark/pull/7740#issuecomment-125799573
Changed from 4 space indentation to 2 space indentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user sabhyankar opened a pull request:
https://github.com/apache/spark/pull/7729
[SPARK-8920] [MLlib] Add @since tags to mllib.linalg
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sabhyankar/spark branch_8920
51 matches
Mail list logo
