[
https://issues.apache.org/jira/browse/SPARK-31604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Divya Paliwal updated SPARK-31604:
----------------------------------
Description:
Hi,
I am currently facing the below error when I run my code to stream data from
Couchbase in spark cluster.
2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in
connection from /[host]:56910
java.lang.IllegalArgumentException: Frame length should be positive:
-9223371863711366549
at
org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119)
at
org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134)
at
org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
When ever I run the spark-submit command with all the arguments on the host I
get the above error.
Command run:
bin/spark-submit \
--deploy-mode cluster \
--class "com.apple.pst.dc.CouchbaseRawMain" \
--master [spark://rn-dcf5t-lapp85.rno.apple.com:8091] \
--jars
/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/core-io-1.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/java-client-2.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxjava-1.3.8.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/couchbase-spark-connector_2.11-2.4.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/dcp-client-0.23.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/opentracing-api-0.31.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxscala_2.11-0.27.0.jar
\
--conf spark.rpc.message.maxSize=1000 —-conf
spark.shuffle.service.enabled=true —-conf
spark.network.crypto.saslFallback=true —-conf spark.authenticate=true —-conf
spark.network.crypto.enabled=true
--driver-memory 3g --driver-cores 2 --num-executors 1 --executor-memory 3g
--total-executor-cores 2 --executor-cores 2 \
/ngs/app/dcf5d/HD-cluster/spark/code/generic-couchbase-raw-0.1.0-SNAPSHOT.jar
[spark://rn-dcf5t-lapp85.rno.apple.com:8091] Couchbase-DC-Bucket-Raw-QA-OG-28
rn-dcf5t-lapp87.rno.apple.com:8091,rn-dcf5t-lapp88.rno.apple.com:8091 DC
[hdfs://rn-dcf5t-lapp85.rno.apple.com:9000/tables/couchbase/QA/DC-bucket/]
Administrator dcadmin welcome true
Thanks,
Divya
was:
I am currently facing the below error when i run my code of streaming data from
couchbase in spark master cluster.
2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in
connection from /[host]:56910
java.lang.IllegalArgumentException: Frame length should be positive:
-9223371863711366549
at
org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119)
at
org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134)
at
org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at
io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)
When ever I run the spark-submit command with all the arguments on the host I
get the above error.
Thanks,
Divya
> java.lang.IllegalArgumentException: Frame length should be positive
> -------------------------------------------------------------------
>
> Key: SPARK-31604
> URL: https://issues.apache.org/jira/browse/SPARK-31604
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 2.4.4
> Environment: Scala version 2.11.12
> Spark version 2.4.4
> Reporter: Divya Paliwal
> Priority: Major
> Fix For: 2.4.4
>
>
> Hi,
> I am currently facing the below error when I run my code to stream data from
> Couchbase in spark cluster.
> 2020-04-29 00:04:06,061 WARN server.TransportChannelHandler: Exception in
> connection from /[host]:56910
> java.lang.IllegalArgumentException: Frame length should be positive:
> -9223371863711366549
> at
> org.spark_project.guava.base.Preconditions.checkArgument(Preconditions.java:119)
> at
> org.apache.spark.network.util.TransportFrameDecoder.decodeNext(TransportFrameDecoder.java:134)
> at
> org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:81)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
> at
> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
> at java.lang.Thread.run(Thread.java:748)
>
> When ever I run the spark-submit command with all the arguments on the host I
> get the above error.
>
> Command run:
> bin/spark-submit \
> --deploy-mode cluster \
> --class "com.apple.pst.dc.CouchbaseRawMain" \
> --master [spark://rn-dcf5t-lapp85.rno.apple.com:8091] \
> --jars
> /ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/core-io-1.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/java-client-2.7.6.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxjava-1.3.8.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/couchbase-spark-connector_2.11-2.4.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/dcp-client-0.23.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/opentracing-api-0.31.0.jar,/ngs/app/dcf5d/HD-cluster/spark/jars/spark-couchbase-scala_2.11/rxscala_2.11-0.27.0.jar
> \
> --conf spark.rpc.message.maxSize=1000 —-conf
> spark.shuffle.service.enabled=true —-conf
> spark.network.crypto.saslFallback=true —-conf spark.authenticate=true —-conf
> spark.network.crypto.enabled=true
> --driver-memory 3g --driver-cores 2 --num-executors 1 --executor-memory 3g
> --total-executor-cores 2 --executor-cores 2 \
>
> /ngs/app/dcf5d/HD-cluster/spark/code/generic-couchbase-raw-0.1.0-SNAPSHOT.jar
> [spark://rn-dcf5t-lapp85.rno.apple.com:8091] Couchbase-DC-Bucket-Raw-QA-OG-28
> rn-dcf5t-lapp87.rno.apple.com:8091,rn-dcf5t-lapp88.rno.apple.com:8091 DC
> [hdfs://rn-dcf5t-lapp85.rno.apple.com:9000/tables/couchbase/QA/DC-bucket/]
> Administrator dcadmin welcome true
>
> Thanks,
> Divya
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
