GitHub user princejwesley opened a pull request:
https://github.com/apache/spark/pull/14502
[SPARK-16909][Spark Core] - Streaming for postgreSQL JDBC driver
As per the postgreSQL JDBC driver
[implementation](https://github.com/pgjdbc/pgjdbc/blob/ab2a6d89081fc2c1fdb2a8600f413db33669022c/pgjdbc/src/main/java/org/postgresql/PGProperty.java#L99),
the default record fetch size is 0(which means, it caches all record)
This fix enforces default record fetch size as 10 to enable streaming of
data.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/princejwesley/spark spark-postgres
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14502.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14502
----
commit c45f8212add3ac7ce04edd0ea1b3903ff9782c6d
Author: Prince J Wesley <[email protected]>
Date: 2016-08-05T03:26:28Z
SPARK-16909 - Streaming for postgreSQL JDBC driver
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
