HI, Currently using Spark 1.2.2, could you please let me know correct results output count which you got it by using JdbcRDDSuite
Regards, Satish Chandra On Wed, Sep 23, 2015 at 4:02 PM, Rishitesh Mishra <rishi80.mis...@gmail.com> wrote: > Which version of Spark you are using ?? I can get correct results using > JdbcRDD. Infact there is a test suite precisely for this (JdbcRDDSuite) . > I changed according to your input and got correct results from this test > suite. > > On Wed, Sep 23, 2015 at 11:00 AM, satish chandra j < > jsatishchan...@gmail.com> wrote: > >> HI All, >> >> JdbcRDD constructor has following parameters, >> >> *JdbcRDD >> <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html#JdbcRDD(org.apache.spark.SparkContext,%20scala.Function0,%20java.lang.String,%20long,%20long,%20int,%20scala.Function1,%20scala.reflect.ClassTag)>* >> (SparkContext >> <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/SparkContext.html> >> sc, >> scala.Function0<java.sql.Connection> getConnection, String sql, *long >> lowerBound, >> long upperBound, int numPartitions*, scala.Function1<java.sql.ResultSet,T >> <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html>> >> mapRow, >> scala.reflect.ClassTag<T >> <https://spark.apache.org/docs/1.2.0/api/java/org/apache/spark/rdd/JdbcRDD.html> >> > evidence$1) >> >> where the below parameters *lowerBound* refers to Lower boundary of >> entire data, *upperBound *refers to Upper boundary of entire data and >> *numPartitions >> *refer to Number of partitions >> >> Source table to which JbdcRDD is fetching data from Oracle DB has more >> than 500 records but its confusing when I tried several executions by >> changing "numPartitions" parameter >> >> LowerBound,UpperBound,numPartitions: Output Count >> >> 0 ,100 ,1 : 100 >> >> 0 ,100 ,2 : 151 >> >> 0 ,100 ,3 : 201 >> >> >> Please help me in understanding the why Output count is 151 if >> numPartitions is 2 and Output count is 201 if numPartitions is 3 >> >> Regards, >> >> Satish Chandra >> > >
