GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/8343
SPARK-8952 - Restrict invocation of normalizePath()
This is based on @davies comment on SPARK-8952 which suggests to only call
normalizePath() when path starts with '~'
You can merge this pull
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/8101#discussion_r42396001
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/jdbc/PostgresIntegrationSuite.scala
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/8343#issuecomment-135572901
Thanks for the comments @shivaram and @sun-rui. I have standardized in
using supressWarnings for all invocations of normalizePath.
---
If your project is set up
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/8393#issuecomment-135583856
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/8393#issuecomment-135800505
Looks like there is a DB2 Express version that is free, let me see if I can
make a docker image with that.
---
If your project is set up for it, you can reply
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/8558
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/8558#issuecomment-136778964
I'll let @srowen submit his PR, easier I guess :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/8557#issuecomment-136780642
LGTM. I have also closed my PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/8558
[SPARK-10398] Migrate to new lua download scripts
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-10398
Alternatively
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/8774
[SPARK-10511][BUILD] Reset git repository before packaging source distro
The calculation of Spark version is downloading
Scala and Zinc in the build directory which is
inflating the size
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9162#issuecomment-160773129
I have a wip pr #9893, and an open question that I think @JoshRosen could
help about the JDBC drivers for the other docker tests. Anyway, these two PRs
are independent
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-165282872
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-165297878
@JoshRosen , this is currently failing due to missing dependency, could you
help add the DB2 jars on the jenkins slaves ? The directions is available in
the pom.xml
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/9893
[SPARK-10521][SQL][WIP] Utilize Docker for test DB2 JDBS Dialect support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/9892
[SPARK-11910][STREAMING][DOCS] Update twitter4j dependency version
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-11910
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9892#issuecomment-158779843
@srowen Sounds good does it look better now ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/9162
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-180593895
Thanks @shaneknapp , I'll refrain the final say on the downtime to others
on the PMC ( @JoshRosen and others)
---
If your project is set up for it, you can reply
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/9893
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11139
[SPARK-13248][STREAMING] Remove deprecated Streaming APIs.
Remove deprecated Streaming APIs and adjust sample applications.
You can merge this pull request into a Git repository by running
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11092
[SPARK-13189] Cleanup build references to Scala 2.10
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13189
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11077
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11075#issuecomment-180486122
@rxin this is now merged into #11092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11076#issuecomment-180486204
@rxin this is now merged into #11092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11074
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11076
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11075
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11077#issuecomment-180486261
@rxin this is now merged into #11092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11092#issuecomment-180493015
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11074#issuecomment-180486063
@rxin this is now merged into #11092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11076
[SPARK-13193][BUILD] Update docker test image to use Scala 2.11
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13193
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11074
[SPARK-13191] Update Scala 2.11 related dependencies
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13191
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11073
[SPARK-13189][BUILD] Update pom.xml to reference Scala 2.11
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13189
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11077
[SPARK-13194][BUILD] Update release audit tools to use Scala 2.11
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13194
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11092#discussion_r52065716
--- Diff: docker/spark-test/base/Dockerfile ---
@@ -25,7 +25,7 @@ RUN apt-get update && \
apt-get install -y less openjdk-7-jre-headless n
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11073
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11073#issuecomment-179840754
Wrong jira
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11075
[SPARK-13190][BUILD] Update pom.xml to reference Scala 2.11
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lresende/spark SPARK-13190
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11139#discussion_r52381133
--- Diff:
examples/src/main/java/org/apache/spark/examples/streaming/JavaRecoverableNetworkWordCount.java
---
@@ -204,13 +202,25 @@ public static void
GitHub user lresende reopened a pull request:
https://github.com/apache/spark/pull/9893
[SPARK-10521][SQL] Utilize Docker for test DB2 JDBC Dialect support
Add integration tests based on docker to test DB2 JDBC dialect support
You can merge this pull request into a Git repository
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-178250790
@JoshRosen Could you please help with pulling the docker image to the
jenkins slaves as you did with the other mySQL and Postgress ones ?
---
If your project is set
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-178783918
@JoshRosen Strange, it's actually publicly published at
https://hub.docker.com/r/lresende/db2express-c/tags/
And I can download it with the command below from
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-175823938
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/9893#discussion_r51047515
--- Diff: docker-integration-tests/pom.xml ---
@@ -34,6 +34,13 @@
docker-integration-tests
+
+
+ db2
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-174039005
@JoshRosen For this PR to work, there is a need to manually deploy the JDBC
driver as described on the pom. This is similar to the requirement that in
order to run
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-174070205
Jenkins, this is ok to test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-184783167
Ping @tdas @mateiz or any other streaming maintainer ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-184801086
@srowen have just pushed all the style issues.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-184811230
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-185326608
@JoshRosen @shaneknapp I believe this is ready, but Jenkins seems to be
having timeout issues probably when connecting to the db2 docker image. Is
there a way for any
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11139#discussion_r53208178
--- Diff:
examples/src/main/java/org/apache/spark/examples/streaming/JavaRecoverableNetworkWordCount.java
---
@@ -204,13 +202,17 @@ public static void
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-185382474
Ok, looks like the issue is that, when trying to start the docker db2
image, we are getting an memory error: SQL1220N The database manager failed to
allocate shared
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-186560354
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-194983724
@rxin Updated the PR with more details
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-194984207
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-198151304
@rxin Please review this and merge/close the JIRA as I have checked with
author and having the fix on 2.0 is ok (so no need to backport it).
---
If your project
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11623
[SPARK-12555][SQL] Add provided test scenario
## What changes were proposed in this pull request?
This PR add test case described in this issue to avoid future regressions.
This test
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-206479123
@cloud-fan Tested the latest commit in both 1.6 branch and trunk... fails
in 1.6 and works fine in trunk.
---
If your project is set up for it, you can reply
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12349
[SPARK-14590] Update pull request template with JIRA link
## What changes were proposed in this pull request?
Update pull request template to have direct link to jira issue
You can
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12349#issuecomment-209221983
@rxin It is, but you then have to search for the jira. This makes it much
more easy to just access the jira, from the link, similar to how the jira has
the link
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12349#issuecomment-209225537
ok, I will leave it open for a day in case anyone else has interest on the
change, otherwise close it.
---
If your project is set up for it, you can reply
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12348
[SPARK-14589][SQL] Enhance DB2 JDBC Dialect docker tests
## What changes were proposed in this pull request?
Enhance the DB2 JDBC Dialect docker tests as they seemed to have had some
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207676407
@JoshRosen, thanks for reviewing, I agree with your proposal regarding the
repository. Thanks.
---
If your project is set up for it, you can reply to this email
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12269
[SPARK-13419] Update SubquerySuite to use checkAnswer to validate resâ¦
## What changes were proposed in this pull request?
Change SubquerySuite to validate test results utilizing
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12270
[SPARK-14504] Enable Oracle docker tests
## What changes were proposed in this pull request?
Enable Oracle docker tests
## How was this patch tested?
Existing tests
You
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-207706296
This PR is dependent on PR #9893
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207697108
@JoshRosen All good, style regression fixed, all tests passing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/12265#discussion_r59119361
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala ---
@@ -152,17 +152,17 @@ class CountVectorizer(override val uid: String
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12265#issuecomment-207827549
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207675856
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58633691
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r57330036
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -206,4 +216,12 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-200962644
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r57098172
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -206,4 +216,12 @@ class DatasetAggregatorSuite extends QueryTest
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11542
[SPARK-13703][BUILD] Remove obsolete scala-2.10 source files
## What changes were proposed in this pull request?
Remove obsolete scala-2.10 source files
## How was this patch
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-192938438
@srowen I was under the impression we were not supporting 2.10 anymore, if
this is not the case I will close this PR.
---
If your project is set up for it, you can
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58480578
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -84,6 +84,16 @@ object ComplexBufferAgg extends Aggregator
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58480607
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-205629190
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58505416
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58487110
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -84,6 +84,16 @@ object ComplexBufferAgg extends Aggregator
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-205413804
@JoshRosen could you please review this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-205420648
@cloud-fan could you please review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12151
[SPARK-14366] Remove sbt-idea plugin
## What changes were proposed in this pull request?
Remove sbt-idea plugin as importing sbt project provides much better
support.
You can merge
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203179609
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203183327
@markhamstra I had done this is the past, and with the current discussion
of droping jdk 7 and scala 2.10 i think this might be useful now. Any concerns
reopening
GitHub user lresende reopened a pull request:
https://github.com/apache/spark/pull/11542
[SPARK-13703][BUILD] Remove obsolete scala-2.10 source files
## What changes were proposed in this pull request?
Remove obsolete scala-2.10 source files
## How was this patch
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203206216
@markhamstra The discussion seems to be getting consensus, so it's not that
I am jumping the gun and creating something completely new, but reopening
something
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-209744652
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/12349
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-209597909
@JoshRosen please review, these are the merge issues I mentioned yesterday,
and these changes are passing my local tests as well.
---
If your project is set up
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12508#issuecomment-214993748
@vanzin @srowen I wouldn't recommend running when docker is installed, as
this would run on the Jenkins slaves, and @JoshRosen disable these because they
were failing
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12705
[SPARK-12660][SQL][WIP] Rewrite except using anti-join
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12705#issuecomment-214833015
@hvanhovell thanks for the comments, this is a wip and i should have a
better working code in a few hours.
---
If your project is set up for it, you can reply
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/12705
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/13229#issuecomment-220753618
LGTM, I also verified with a issue discussed on the mailing list where
writing to csv was taking about 1hr, and with this change it only took a few
minutes
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/9162#discussion_r64446518
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
---
@@ -579,6 +579,14 @@ class JDBCSuite extends SparkFunSuite
val
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9162#issuecomment-221358898
Other then the minor comment around BooleanType, it LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
1 - 100 of 147 matches
Mail list logo
