Hi, Could you provide more details about your queries and setup? Logs could be helpful as well.
Piotrek > On 9 Mar 2018, at 11:00, 杨力 <bill.le...@gmail.com> wrote: > > I wrote a flink-sql app with following topography. > > KafkaJsonTableSource -> SQL -> toAppendStream -> Map -> JDBCAppendTableSink > KafkaJsonTableSource -> SQL -> toAppendStream -> Map -> JDBCAppendTableSink > ... > KafkaJsonTableSource -> SQL -> toAppendStream -> Map -> JDBCAppendTableSink > > I have a dozen of TableSources And tens of SQLs. As a result, the number of > JDBCAppendTableSink times parallelism, that is the number of concurrent > connections to database, is too large for the database server to handle. So I > tried union DataStreams before connecting them to the TableSink. > > KafkaJsonTableSource -> SQL -> toAppendStream -> Map > \ > KafkaJsonTableSource -> SQL -> toAppendStream -> Map --- union -> > JDBCAppendTableSink > ... / > KafkaJsonTableSource -> SQL -> toAppendStream -> Map > > With this strategy, job submission failed with an OversizedPayloadException > of 104 MB. Increasing akka.framesize helps to avoid this exception, but job > submission hangs and times out. > > I can't understand why a simple union operator would serialize to such a > large message. Can I avoid this problem? > Or can I change some configuration to fix the submission time out? > > Regards, > Bill
