RE: Spark application fail wit numRecords error

2017-11-01 Thread Serkan TAS 
Hi,

I checked the following threads but i am still not sure if it is misuse, common 
o a bug.

https://stackoverflow.com/questions/34989539/spark-streaming-from-kafka-has-error-numrecords-must-not-be-negative

https://stackoverflow.com/questions/41319530/why-does-spark-streaming-application-with-kafka-fail-with-requirement-failed-n

https://forums.databricks.com/questions/11055/how-to-resolve-illegalargumentexception-requiremen.html



From: Prem Sure [mailto:sparksure...@gmail.com]
Sent: Wednesday, November 1, 2017 8:11 PM
To: Serkan TAS <serkan@enerjisa.com>
Cc: user@spark.apache.org
Subject: Re: Spark application fail wit numRecords error

Hi, any offset left over for new topic consumption?, case can be the offset is 
beyond current latest offset and cuasing negative.
hoping kafka brokers health is good and are up, this can also be a reason 
sometimes.

On Wed, Nov 1, 2017 at 11:40 AM, Serkan TAS 
<serkan@enerjisa.com<mailto:serkan@enerjisa.com>> wrote:
Hi,

I searched the error in kafka but i think at last, it is related with spark not 
kafka.

Has anyone faced to an exception that is terminating program with error 
"numRecords must not be negative" while streaming  ?

Thanx in advance.

Regards.



Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler 
içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve 
dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene 
telefonla ya da e-posta ile bildirmeli ve bilgisayarından silmelidir. Bu 
iletinin içeriğinden yalnızca iletiyi gönderen kişi sorumludur.

This communication may contain information that is legally privileged, 
confidential or exempt from disclosure. If you are not the intended recipient, 
please note that any dissemination, distribution, or copying of this 
communication is strictly prohibited. Anyone who receives this message in error 
should notify the sender immediately by telephone or by return communication 
and delete it from his or her computer. Only the person who has sent this 
message is responsible for its content.




Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler 
içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve 
dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene 
telefonla ya da e-posta ile bildirmeli ve bilgisayarından silmelidir. Bu 
iletinin içeriğinden yalnızca iletiyi gönderen kişi sorumludur.

This communication may contain information that is legally privileged, 
confidential or exempt from disclosure. If you are not the intended recipient, 
please note that any dissemination, distribution, or copying of this 
communication is strictly prohibited. Anyone who receives this message in error 
should notify the sender immediately by telephone or by return communication 
and delete it from his or her computer. Only the person who has sent this 
message is responsible for its content.

Re: Spark application fail wit numRecords error

2017-11-01 Thread Prem Sure 
Hi, any offset left over for new topic consumption?, case can be the offset
is beyond current latest offset and cuasing negative.
hoping kafka brokers health is good and are up, this can also be a reason
sometimes.

On Wed, Nov 1, 2017 at 11:40 AM, Serkan TAS  wrote:

> Hi,
>
>
>
> I searched the error in kafka but i think at last, it is related with
> spark not kafka.
>
>
>
> Has anyone faced to an exception that is terminating program with error
> "numRecords must not be negative" while streaming  ?
>
>
>
> Thanx in advance.
>
>
>
> Regards.
>
> --
>
> Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler
> içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi
> çoğaltmak ve dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu
> derhal gönderene telefonla ya da e-posta ile bildirmeli ve bilgisayarından
> silmelidir. Bu iletinin içeriğinden yalnızca iletiyi gönderen kişi
> sorumludur.
>
> This communication may contain information that is legally privileged,
> confidential or exempt from disclosure. If you are not the intended
> recipient, please note that any dissemination, distribution, or copying of
> this communication is strictly prohibited. Anyone who receives this message
> in error should notify the sender immediately by telephone or by return
> communication and delete it from his or her computer. Only the person who
> has sent this message is responsible for its content.
>

Spark application fail wit numRecords error

2017-11-01 Thread Serkan TAS 
Hi,

I searched the error in kafka but i think at last, it is related with spark not 
kafka.

Has anyone faced to an exception that is terminating program with error 
"numRecords must not be negative" while streaming  ?

Thanx in advance.

Regards.



Bu ileti hukuken korunmuş, gizli veya ifşa edilmemesi gereken bilgiler 
içerebilir. Şayet mesajın gönderildiği kişi değilseniz, bu iletiyi çoğaltmak ve 
dağıtmak yasaktır. Bu mesajı yanlışlıkla alan kişi, bu durumu derhal gönderene 
telefonla ya da e-posta ile bildirmeli ve bilgisayarından silmelidir. Bu 
iletinin içeriğinden yalnızca iletiyi gönderen kişi sorumludur.

This communication may contain information that is legally privileged, 
confidential or exempt from disclosure. If you are not the intended recipient, 
please note that any dissemination, distribution, or copying of this 
communication is strictly prohibited. Anyone who receives this message in error 
should notify the sender immediately by telephone or by return communication 
and delete it from his or her computer. Only the person who has sent this 
message is responsible for its content.