Hello Dev Users, I am struggling to parallelize JDBC Read in Spark, It is using 1 - 2 task only to read data and taking so much of time to read.
Ex. val invoiceLineItemDF = ((spark.read.jdbc(url = t360jdbcURL, table = invoiceLineItemQuery, columnName = "INVOICE_LINE_ITEM_ID", lowerBound = 1L, upperBound = 1000000L, numPartitions = 200, connectionProperties = connectionProperties ))) Thanks