GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/18603
[SPARK-21370][SS] Add test for state reliability when one read-only state
store aborts after read-write state store commits
## What changes were proposed in this pull request?
During
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/18660
[SPARK-21445] Make IntWrapper and LongWrapper in UTF8String Serializable
## What changes were proposed in this pull request?
Making those two classes will avoid Serialization issues like
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18660
I don't think we're actually trying to ship these values anywhere. We're
failing to serialize the task during WholeStageCodegenExec:
```
at
org.apache.spark.util
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18660
cc @cloud-fan and @tejasapatil who have more info based on:
https://github.com/apache/spark/commit/c96d14abae5962a7b15239319c2a151b95f7db94
---
If your project is set up for it, you can reply
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18660
I couldn't write an easy reproduction for the bug :(
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18660
thanks @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18660
Also merged to branch-2.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/18676
[SPARK-21463] Allow userSpecifiedSchema to override partition inference
performed by MetadataLogFileIndex
## What changes were proposed in this pull request?
When using the
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18676
One thing that worries me is the fact that we're paying the price of
inferring partitions and reading all files from the log twice. I tried re-using
the `metadataLog` instance in the copy of th
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/18676
Thanks! Merging to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
Jenkins add to whitelist
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268999
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala
---
@@ -81,22 +84,35 @@ class MemorySinkV2 extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195269434
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/memoryV2.scala
---
@@ -110,40 +126,61 @@ class MemorySinkV2 extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268218
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268299
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195268861
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -228,19 +229,45 @@ trait MemorySinkBase extends
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195797571
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -221,26 +222,72 @@ class MemoryStreamInputPartition(records
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/21559#discussion_r195798990
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/memory.scala
---
@@ -221,26 +222,72 @@ class MemoryStreamInputPartition(records
Github user brkyvz commented on the issue:
https://github.com/apache/spark/pull/21559
Thanks! Merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/5616
[SPARK-6829] Added math functions for DataFrames
Implemented code-gen and functions for almost all math functions found in
scala.math (max, min and abs were already present).
cc @mengxr
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23118561
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4109
[SPARK-5321] Support for transposing local matrices
Support for transposing local matrices added. The `.transpose` function
creates a new object re-using the backing array(s) but switches `numRows
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23241236
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,223 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23271028
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23271097
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23271303
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23271456
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4109#discussion_r23278231
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -114,21 +106,47 @@ class DenseMatrix(val numRows: Int, val numCols: Int
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4109#discussion_r23573354
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -92,6 +84,16 @@ sealed trait Matrix extends Serializable
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4215
[WIP][SPARK-5341] Use maven coordinates as dependencies in spark-shell and
spark-submit
This PR adds support for using maven coordinates as dependencies to
spark-shell.
Coordinates can be
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4109#discussion_r23583664
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -34,6 +34,9 @@ sealed trait Matrix extends Serializable {
/** Number
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r23589329
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4109#discussion_r23591427
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -212,6 +284,17 @@ class MatricesSuite extends FunSuite
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/3200#issuecomment-71859402
@mengxr I don't know if `rows` and `cols` will be confusing in terms of
naming in GridPartitioner...
However, since it is private and internal, maybe it'
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-71929196
@pwendell @mateiz I think the PR is ready for code review. I would
appreciate your comments!
---
If your project is set up for it, you can reply to this email and have
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4256
[SPARK-3977] Conversion methods for BlockMatrix to other Distributed
Matrices
The conversion methods for `BlockMatrix`. Conversions go through
`CoordinateMatrix` in order to cause a shuffle so that
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4274
[SPARK-3975] Added support for BlockMatrix addition and multiplication
Support for multiplying and adding large distributed matrices!
You can merge this pull request into a Git repository by running
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4275
[SPARK-5322] Added transpose functionality to BlockMatrix
BlockMatrices can now be transposed!
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4279
[SPARK-5486] Added validate method to BlockMatrix
The `validate` method will allow users to debug their `BlockMatrix`, if
operations like `add` or `multiply` return unexpected results. It checks the
Github user brkyvz closed the pull request at:
https://github.com/apache/spark/pull/2451
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/2451#issuecomment-72128781
closing this PR as a lot of functionality has changed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4286
[SPARK-3976] Added repartitioning for BlockMatrix
Allows users to repartition their BlockMatrices into blocks with different
dimensions
You can merge this pull request into a Git repository by
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4652#issuecomment-74727279
@pwendell this is not enough to support Spark Packages with pyspark but
solves the harder half of the problem. I have a follow up patch that adds jars
in `--packages` to
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4662
[SPARK-5811] Added documentation for maven coordinates and added Spark
Packages support
Documentation for maven coordinates + Spark Package support. Added pyspark
tests for `--packages`
You can
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4662#issuecomment-74770805
Adding spark-packages as a repo broke one test. Pushing update once I make
sure it passes it locally.
---
If your project is set up for it, you can reply to this email
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4664
[SPARK-5507] Added documentation for BlockMatrix
Docs for BlockMatrix. @mengxr
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/brkyvz/spark
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4737#issuecomment-75684685
LGTM. I thought it would be nice to show how people can go back to
`RowMatrix` and call SVD after some operations, but we can keep it simple. I
apologize for missing the
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4754
[SPARK-5979] Made --package exclusions more refined
@tdas Would you care to try this? I think it should solve your problem
You can merge this pull request into a Git repository by running
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-75912004
@tdas added a hack to include the jars on --driver-extra-classpath. Can you
try your test now?
---
If your project is set up for it, you can reply to this email and have
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76019148
@tdas I think that's returning an error now, because the class in fact
doesn't really exist in the jar. Not because it's not in the classpath
---
If your
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76019805
nvm, it should be in spark-streaming-kafka_2.10.jar
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76108899
@tdas @pwendell @andrewor14
This is ready for code review. Moved the resolve method to
DriverBootstrapper. In case the DriverBootstrapper is not called, the
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76118756
It might not be a flaky test. I might have broken some Yarn feature. I'm
going to check once I get home.
On Feb 25, 2015 8:01 PM, "Tathagata D
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76123576
@tdas The latest commit fixed the issue, feel free to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76215606
This passed locally. What the...
On Feb 26, 2015 8:39 AM, "UCB AMPLab" wrote:
> Test FAILed.
> Refer to this link for build results (acc
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76289777
Flaky test this time... @tdas, can you have this retested please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4754#issuecomment-76304574
@srowen Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/4802
[SPARK-5979][SPARK-6032] Smaller safer fix
@pwendell @tdas
This is the safer parts of PR #4754
You can merge this pull request into a Git repository by running:
$ git pull https
Github user brkyvz closed the pull request at:
https://github.com/apache/spark/pull/4754
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r21881450
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -313,15 +593,145 @@ object Matrices {
* @return Square `DenseMatrix
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r21881510
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -197,6 +295,171 @@ class SparseMatrix(
}
override def
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r21881678
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -197,6 +295,171 @@ class SparseMatrix(
}
override def
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r21929684
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -313,15 +593,145 @@ object Matrices {
* @return Square `DenseMatrix
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r22026283
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -256,72 +524,297 @@ object Matrices {
* Generate a `DenseMatrix
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3319#discussion_r22094195
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
---
@@ -197,6 +331,145 @@ class SparseMatrix(
}
override def
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/3200
[SPARK-3974][MLlib] Distributed Block Matrix Abstractions
This pull request includes the abstractions for the distributed BlockMatrix
representation.
`BlockMatrix` will allow users to store
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/3200#issuecomment-62982228
@mengxr
> If we have two block matrices, A and B, and A's column block partitioning
matches B's row block partitioning, can we take advantage
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3200#discussion_r20378789
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/3319
[SPARK-4409][MLlib] Additional Linear Algebra Utils
Addition of a very limited number of local matrix manipulation and
generation methods that would be helpful in the further development for
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/3319#issuecomment-64505065
@mengxr:
Thanks for the feedback. Added the Java tests!
horzcat and vertcat are in fact MATLAB methods:
http://www.mathworks.com/help/matlab/ref/horzcat.html
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/3468#discussion_r20916331
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala ---
@@ -112,4 +116,50 @@ class MatricesSuite extends FunSuite
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/3468#issuecomment-64515054
Looks good to me! Just made one comment, no biggie though, it's fine as is
(but if you decide to change it, there are 4 exact copies of it).
One comment/question
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/3319#issuecomment-64713893
I checked MATLAB's webpage, I didn't see any copyright mentions for the
method names. It's best to triple check though. Since numPy and sciPy share
me
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4279#discussion_r23828653
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -172,6 +172,64 @@ class BlockMatrix(
assert(cols
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23879232
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -431,6 +458,155 @@ object SparkSubmit {
}
}
+/** Provides
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-72283499
I will add documentation during the QA period
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23879289
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitUtilsSuite.scala ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23879684
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -123,6 +126,7 @@ private[spark] class SparkSubmitArguments(args:
Seq
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23877788
--- Diff: core/pom.xml ---
@@ -225,6 +225,16 @@
test
+ org.apache.ivy
+ ivy
+ ${ivy.version
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-72282474
@JoshRosen thank you very much for the time and comments. I'll fix things
immediately
---
If your project is set up for it, you can reply to this email and have
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4274#discussion_r23862043
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/BlockMatrix.scala
---
@@ -237,4 +239,88 @@ class BlockMatrix(
val localMat
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-72326441
Interesting... The tests are successful on my local computer but fails in
Jenkins... The end to end test that downloads spark-avro and spark-csv succeeds
which is nice
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23972143
--- Diff:
core/src/test/scala/org/apache/spark/deploy/SparkSubmitUtilsSuite.scala ---
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-72598548
@JoshRosen I actually don't know what we are missing. I think it only
requires testing, because the directory structure (backslashes instead of
slashes) and command
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r23985067
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -475,6 +501,195 @@ object SparkSubmit {
}
}
+/** Provides
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4215#issuecomment-72674385
@pwendell, I think this is in good shape to go in right before you cut the
branch. Having the community test it out under many different settings and
setups would help a
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r24047115
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -25,6 +25,18 @@ import scala.collection.mutable.{ArrayBuffer, HashMap,
Map
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/4215#discussion_r24047567
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -25,6 +25,18 @@ import scala.collection.mutable.{ArrayBuffer, HashMap,
Map
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4447#issuecomment-73355639
LGTM, I'm perfectly okay with this. Put those there before just in case
---
If your project is set up for it, you can reply to this email and have your
reply appe
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/4603#issuecomment-74388179
This fixes the problem! looks good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/5616#discussion_r29195806
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathfunctions.scala
---
@@ -0,0 +1,215 @@
+/*
+ * Licensed to the
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/5616#discussion_r29213354
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala
---
@@ -83,8 +83,8 @@ abstract class BinaryArithmetic
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/5616#discussion_r29213390
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -643,4 +644,26 @@ trait HiveTypeCoercion
Github user brkyvz commented on the pull request:
https://github.com/apache/spark/pull/5616#issuecomment-96935194
Sure @rxin let me know what they are. I'll submit the PR for Python
tomorrow.
---
If your project is set up for it, you can reply to this email and have your
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/5750
[SPARK-7188] added python support for math DataFrame functions
Adds support for the math functions for DataFrames in PySpark.
@rxin I love Davies.
You can merge this pull request into a
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/5755
[SPARK-7205] Support `.ivy2/local` and `.m2/repositories/` in --packages
In addition, I made a small change that will allow users to import 2
different artifacts with the same name. That change is
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/5761
[SPARK-7156][SQL] support RandomSplit in DataFrames
This is built on top of @kaka1992 's PR #5711 using Logical plans.
You can merge this pull request into a Git repository by running:
GitHub user brkyvz opened a pull request:
https://github.com/apache/spark/pull/5762
[SPARK-7215] made coalesce and repartition a part of the query plan
Coalesce and repartition now show up as part of the query plan, rather than
resulting in a new `DataFrame`.
You can merge this
Github user brkyvz commented on a diff in the pull request:
https://github.com/apache/spark/pull/5762#discussion_r29301550
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/basicOperators.scala ---
@@ -245,6 +245,19 @@ case class Distinct(partial: Boolean, child
501 - 600 of 1226 matches
Mail list logo
