Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/15966
Thank you for review, @cloud-fan .
With the same parameter name `numPartitions` for read/write, we will use
the same parallelism by default. It's easy to use.
To use different parallelisms for read/write, we are able to do that with
different view names. In the following example, t1 is `numPartitions=1` and t2
is `numPartitions=2`. It's an example for writing, but I think the situation is
the same with read operation.
```scala
sql("CREATE OR REPLACE TEMPORARY VIEW data USING org.apache.spark.sql.jdbc
OPTIONS (url 'jdbc:mysql://localhost:3306/t', dbtable 'data', user 'root',
password '')")
sql("CREATE OR REPLACE TEMPORARY VIEW t1 USING org.apache.spark.sql.jdbc
OPTIONS (url 'jdbc:mysql://localhost:3306/t', dbtable 't1', user 'root',
password '', numPartitions '1')")
sql("CREATE OR REPLACE TEMPORARY VIEW t2 USING org.apache.spark.sql.jdbc
OPTIONS (url 'jdbc:mysql://localhost:3306/t', dbtable 't1', user 'root',
password '', numPartitions '2')")
sql("INSERT OVERWRITE TABLE t1 SELECT a FROM data GROUP BY a")
sql("INSERT OVERWRITE TABLE t2 SELECT a FROM data GROUP BY a")
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
