The weight field is not nullable. Looks like your source table had null value for this field.
On Wed, Apr 20, 2016 at 4:11 PM, Charles Nnamdi Akalugwu < cprenzb...@gmail.com> wrote: > Hi, > > I am using spark 1.4.1 and trying to copy all rows from a table in one > MySQL Database to a Amazon RDS table using spark SQL. > > Some columns in the source table are defined as DECIMAL type and are > nullable. Others are not. When I run my spark job, > > val writeData = sqlContext.read.format("jdbc").option("url", >>> sourceUrl).option("driver", "com.mysql.jdbc.Driver").option("dbtable", >>> table).option("user", sourceUsername).option("password", >>> sourcePassword).load() >> >> >> >> >> >> writeData.write.format("com.databricks.spark.redshift").option("url", >>> String.format(targetUrl, targetUsername, targetPassword)).option("dbtable", >>> table).option("tempdir", redshiftTempDir+table).mode("append").save() >> >> > it fails with the following exception > > Can't translate null value for field >> StructField(weight,DecimalType(5,2),false) > > > Any insights about this exception would be very helpful. >