Hi Sunny,
As stated by Fabian try to see whether including the postgres classes in
the shaded jar solves the problem. If it doesn't, you're probably hitting
the same problem i had with an older version of Flink (
https://issues.apache.org/jira/plugins/servlet/mobile#issue/FLINK-4061) and
this you have to copy the postgres jar in the Flink lib directory.
Best,
Flavio
On 12 Oct 2016 23:36, "Fabian Hueske" <fhue...@gmail.com> wrote:
> Hi Sunny,
>
> please avoid crossposting to all mailing lists.
> The dev@f.a.o list is for issues related to the development of Flink not
> the development of Flink applications.
>
> The error message is actually quite descriptive. Flink does not find the
> JDBC driver class.
> You need to add it to the classpath for example by adding the
> corresponding Maven dependency to your pom file.
>
> Fabian
>
>
> 2016-10-12 23:18 GMT+02:00 sunny patel <sunnyle...@gmail.com>:
>
>>
>> Hi Guys,
>>
>> I am facing JDBC error, could you please some one advise me on this error?
>>
>> $ java -version
>>
>> java version "1.8.0_102"
>>
>> Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
>>
>> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
>>
>> $ scala -version
>>
>> Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
>>
>>
>> =============== Scala Code
>>
>> import org.apache.flink.api.common.typeinfo.TypeInformation
>> import org.apache.flink.api.java.io.jdbc.JDBCInputFormat
>> import org.apache.flink.api.scala._
>> import org.apache.flink.api.table.typeutils.RowTypeInfo
>>
>> object WordCount {
>> def main(args: Array[String]) {
>>
>> val PATH = getClass.getResource("").getPath
>>
>> // set up the execution environment
>> val env = ExecutionEnvironment.getExecutionEnvironment
>>
>> // Read data from JDBC (Kylin in our case)
>> val stringColum: TypeInformation[Int] = createTypeInformation[Int]
>> val DB_ROWTYPE = new RowTypeInfo(Seq(stringColum))
>>
>> val inputFormat = JDBCInputFormat.buildJDBCInputFormat()
>> .setDrivername("org.postgresql.jdbc.Driver")
>> .setDBUrl("jdbc:postgresql://localhost:5432/mydb")
>> .setUsername("MI")
>> .setPassword("MI")
>> .setQuery("select * FROM identity")
>> .setRowTypeInfo(DB_ROWTYPE)
>> .finish()
>>
>> val dataset =env.createInput(inputFormat)
>> dataset.print()
>>
>> println(PATH)
>> }
>> }
>>
>> ==========================================================================
>>
>> ==========POM.XML
>>
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <project xmlns="http://maven.apache.org/POM/4.0.0";
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>> http://maven.apache.org/xsd/maven-4.0.0.xsd";>
>> <modelVersion>4.0.0</modelVersion>
>> <parent>
>> <artifactId>flink-parent</artifactId>
>> <groupId>org.apache.flink</groupId>
>> <version>1.2-SNAPSHOT</version>
>> </parent>
>>
>> <groupId>org.apache.flink.quickstart</groupId>
>> <artifactId>flink-scala-project</artifactId>
>> <version>0.1</version>
>> <packaging>jar</packaging>
>>
>> <name>Flink Quickstart Job</name>
>> <url>http://www.myorganization.org</url>
>>
>> <repositories>
>> <repository>
>> <id>apache.snapshots</id>
>> <name>Apache Development Snapshot Repository</name>
>>
>> <url>https://repository.apache.org/content/repositories/snapshots/</url>
>> <releases>
>> <enabled>false</enabled>
>> </releases>
>> <snapshots>
>> </snapshots>
>> </repository>
>> </repositories>
>>
>> <properties>
>> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
>> <flink.version>1.1.2</flink.version>
>> </properties>
>>
>> <!--
>>
>> Execute "mvn clean package -Pbuild-jar"
>> to build a jar file out of this project!
>>
>> How to use the Flink Quickstart pom:
>>
>> a) Adding new dependencies:
>> You can add dependencies to the list below.
>> Please check if the maven-shade-plugin below is filtering out your
>> dependency
>> and remove the exclude from there.
>>
>> b) Build a jar for running on the cluster:
>> There are two options for creating a jar from this project
>>
>> b.1) "mvn clean package" -> this will create a fat jar which
>> contains all
>> dependencies necessary for running the jar created by this
>> pom in a cluster.
>> The "maven-shade-plugin" excludes everything that is
>> provided on a running Flink cluster.
>>
>> b.2) "mvn clean package -Pbuild-jar" -> This will also create a
>> fat-jar, but with much
>> nicer dependency exclusion handling. This approach is
>> preferred and leads to
>> much cleaner jar files.
>> -->
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-jdbc</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-table_2.11</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-scala_2.11</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-streaming-scala_2.11</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-clients_2.11</artifactId>
>> <version>${flink.version}</version>
>> </dependency>
>> </dependencies>
>>
>> <profiles>
>> <profile>
>> <!-- Profile for packaging correct JAR files -->
>> <id>build-jar</id>
>> <activation>
>> </activation>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-scala_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-streaming-scala_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.flink</groupId>
>> <artifactId>flink-clients_2.11</artifactId>
>> <version>${flink.version}</version>
>> <scope>provided</scope>
>> </dependency>
>> </dependencies>
>>
>> <build>
>> <plugins>
>> <!-- disable the exclusion rules -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <version>2.4.1</version>
>> <executions>
>> <execution>
>> <phase>package</phase>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> <configuration>
>> <artifactSet>
>> <excludes combine.self="override"></excludes>
>> </artifactSet>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>> </plugins>
>> </build>
>> </profile>
>> </profiles>
>>
>> <!-- We use the maven-assembly plugin to create a fat jar that contains
>> all dependencies
>> except flink and its transitive dependencies. The resulting fat-jar
>> can be executed
>> on a cluster. Change the value of Program-Class if your program entry
>> point changes. -->
>> <build>
>> <plugins>
>> <!-- We use the maven-shade plugin to create a fat jar that
>> contains all dependencies
>> except flink and it's transitive dependencies. The resulting
>> fat-jar can be executed
>> on a cluster. Change the value of Program-Class if your program
>> entry point changes. -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <version>2.4.1</version>
>> <executions>
>> <!-- Run shade goal on package phase -->
>> <execution>
>> <phase>package</phase>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> <configuration>
>> <artifactSet>
>> <excludes>
>> <!-- This list contains all dependencies of
>> flink-dist
>> Everything else will be packaged into
>> the fat-jar
>> -->
>>
>> <exclude>org.apache.flink:flink-annotations</exclude>
>>
>> <exclude>org.apache.flink:flink-shaded-hadoop1_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-shaded-hadoop2</exclude>
>>
>> <exclude>org.apache.flink:flink-shaded-curator-recipes</exclude>
>> <exclude>org.apache.flink:flink-core</exclude>
>> <exclude>org.apache.flink:flink-java</exclude>
>>
>> <exclude>org.apache.flink:flink-scala_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-runtime_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-optimizer_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-clients_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-avro_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-examples-batch_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-examples-streaming_2.11</exclude>
>>
>> <exclude>org.apache.flink:flink-streaming-java_2.11</exclude>
>>
>> <!-- Also exclude very big transitive
>> dependencies of Flink
>>
>> WARNING: You have to remove these
>> excludes if your code relies on other
>> versions of these dependencies.
>>
>> -->
>>
>> <exclude>org.scala-lang:scala-library</exclude>
>> <exclude>org.scala-lang:scala-compiler</exclude>
>> <exclude>org.scala-lang:scala-reflect</exclude>
>> <exclude>com.typesafe.akka:akka-actor_*</exclude>
>> <exclude>com.typesafe.akka:akka-remote_*</exclude>
>> <exclude>com.typesafe.akka:akka-slf4j_*</exclude>
>> <exclude>io.netty:netty-all</exclude>
>> <exclude>io.netty:netty</exclude>
>>
>> <exclude>commons-fileupload:commons-fileupload</exclude>
>> <exclude>org.apache.avro:avro</exclude>
>>
>> <exclude>commons-collections:commons-collections</exclude>
>>
>> <exclude>com.thoughtworks.paranamer:paranamer</exclude>
>> <exclude>org.xerial.snappy:snappy-java</exclude>
>>
>> <exclude>org.apache.commons:commons-compress</exclude>
>> <exclude>org.tukaani:xz</exclude>
>> <exclude>com.esotericsoftware.kryo:kryo</exclude>
>>
>> <exclude>com.esotericsoftware.minlog:minlog</exclude>
>> <exclude>org.objenesis:objenesis</exclude>
>> <exclude>com.twitter:chill_*</exclude>
>> <exclude>com.twitter:chill-java</exclude>
>> <exclude>commons-lang:commons-lang</exclude>
>> <exclude>junit:junit</exclude>
>>
>> <exclude>org.apache.commons:commons-lang3</exclude>
>> <exclude>org.slf4j:slf4j-api</exclude>
>> <exclude>org.slf4j:slf4j-log4j12</exclude>
>> <exclude>log4j:log4j</exclude>
>> <exclude>org.apache.commons:commons-math</exclude>
>>
>> <exclude>org.apache.sling:org.apache.sling.commons.json</exclude>
>> <exclude>commons-logging:commons-logging</exclude>
>> <exclude>commons-codec:commons-codec</exclude>
>>
>> <exclude>com.fasterxml.jackson.core:jackson-core</exclude>
>>
>> <exclude>com.fasterxml.jackson.core:jackson-databind</exclude>
>>
>> <exclude>com.fasterxml.jackson.core:jackson-annotations</exclude>
>> <exclude>stax:stax-api</exclude>
>> <exclude>com.typesafe:config</exclude>
>>
>> <exclude>org.uncommons.maths:uncommons-maths</exclude>
>> <exclude>com.github.scopt:scopt_*</exclude>
>> <exclude>commons-io:commons-io</exclude>
>> <exclude>commons-cli:commons-cli</exclude>
>> </excludes>
>> </artifactSet>
>> <filters>
>> <filter>
>> <artifact>org.apache.flink:*</artifact>
>> <excludes>
>> <!-- exclude shaded google but include shaded
>> curator -->
>>
>> <exclude>org/apache/flink/shaded/com/**</exclude>
>> <exclude>web-docs/**</exclude>
>> </excludes>
>> </filter>
>> <filter>
>> <!-- Do not copy the signatures in the META-INF
>> folder.
>> Otherwise, this might cause
>> SecurityExceptions when using the JAR. -->
>> <artifact>*:*</artifact>
>> <excludes>
>> <exclude>META-INF/*.SF</exclude>
>> <exclude>META-INF/*.DSA</exclude>
>> <exclude>META-INF/*.RSA</exclude>
>> </excludes>
>> </filter>
>> </filters>
>> <!-- If you want to use ./bin/flink run <quickstart
>> jar> uncomment the following lines.
>> This will add a Main-Class entry to the manifest
>> file -->
>> <!--
>> <transformers>
>> <transformer
>> implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
>>
>> <mainClass>org.apache.flink.quickstart.StreamingJob</mainClass>
>> </transformer>
>> </transformers>
>> -->
>>
>> <createDependencyReducedPom>false</createDependencyReducedPom>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>>
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-compiler-plugin</artifactId>
>> <version>3.1</version>
>> <configuration>
>> <source>1.7</source>
>> <target>1.7</target>
>> </configuration>
>> </plugin>
>> <plugin>
>> <groupId>net.alchim31.maven</groupId>
>> <artifactId>scala-maven-plugin</artifactId>
>> <version>3.1.4</version>
>> <executions>
>> <execution>
>> <goals>
>> <goal>compile</goal>
>> <goal>testCompile</goal>
>> </goals>
>> </execution>
>> </executions>
>> </plugin>
>>
>> <!-- Eclipse Integration -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-eclipse-plugin</artifactId>
>> <version>2.8</version>
>> <configuration>
>> <downloadSources>true</downloadSources>
>> <projectnatures>
>>
>> <projectnature>org.scala-ide.sdt.core.scalanature</projectnature>
>>
>> <projectnature>org.eclipse.jdt.core.javanature</projectnature>
>> </projectnatures>
>> <buildcommands>
>> <buildcommand>
>>
>> ...
