Yup also works for me on master branch as I've been testing DynamoDB Streams integration. In fact works with latest KCL 1.6.1 also which I was using.
So theKCL version does seem like it could be the issue - somewhere along the line an exception must be getting swallowed. Though the tests should have picked this up? Will dig deeper. — Sent from Mailbox On Thu, Dec 10, 2015 at 11:07 PM, Brian London <brianmlon...@gmail.com> wrote: > Yes, it worked in the 1.6 branch as of commit > db5165246f2888537dd0f3d4c5a515875c7358ed. That makes it much less serious > of an issue, although it would be nice to know what the root cause is to > avoid a regression. > On Thu, Dec 10, 2015 at 4:03 PM Burak Yavuz <brk...@gmail.com> wrote: >> I've noticed this happening when there was some dependency conflicts, and >> it is super hard to debug. >> It seems that the KinesisClientLibrary version in Spark 1.5.2 is 1.3.0, >> but it is 1.2.1 in Spark 1.5.1. >> I feel like that seems to be the problem... >> >> Brian, did you verify that it works with the 1.6.0 branch? >> >> Thanks, >> Burak >> >> On Thu, Dec 10, 2015 at 11:45 AM, Brian London <brianmlon...@gmail.com> >> wrote: >> >>> Nick's symptoms sound identical to mine. I should mention that I just >>> pulled the latest version from github and it seems to be working there. To >>> reproduce: >>> >>> >>> 1. Download spark 1.5.2 from http://spark.apache.org/downloads.html >>> 2. build/mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests >>> clean package >>> 3. build/mvn -Pkinesis-asl -DskipTests clean package >>> 4. Then run simultaneously: >>> 1. bin/run-example streaming.KinesisWordCountASL [Kinesis app name] >>> [Kinesis stream name] [endpoint URL] >>> 2. bin/run-example streaming.KinesisWordProducerASL [Kinesis >>> stream name] [endpoint URL] 100 10 >>> >>> >>> On Thu, Dec 10, 2015 at 2:05 PM Jean-Baptiste Onofré <j...@nanthrax.net> >>> wrote: >>> >>>> Hi Nick, >>>> >>>> Just to be sure: don't you see some ClassCastException in the log ? >>>> >>>> Thanks, >>>> Regards >>>> JB >>>> >>>> On 12/10/2015 07:56 PM, Nick Pentreath wrote: >>>> > Could you provide an example / test case and more detail on what issue >>>> > you're facing? >>>> > >>>> > I've just tested a simple program reading from a dev Kinesis stream and >>>> > using stream.print() to show the records, and it works under 1.5.1 but >>>> > doesn't appear to be working under 1.5.2. >>>> > >>>> > UI for 1.5.2: >>>> > >>>> > Inline image 1 >>>> > >>>> > UI for 1.5.1: >>>> > >>>> > Inline image 2 >>>> > >>>> > On Thu, Dec 10, 2015 at 5:50 PM, Brian London <brianmlon...@gmail.com >>>> > <mailto:brianmlon...@gmail.com>> wrote: >>>> > >>>> > Has anyone managed to run the Kinesis demo in Spark 1.5.2? The >>>> > Kinesis ASL that ships with 1.5.2 appears to not work for me >>>> > although 1.5.1 is fine. I spent some time with Amazon earlier in >>>> the >>>> > week and the only thing we could do to make it work is to change >>>> the >>>> > version to 1.5.1. Can someone please attempt to reproduce before I >>>> > open a JIRA issue for it? >>>> > >>>> > >>>> >>>> -- >>>> Jean-Baptiste Onofré >>>> jbono...@apache.org >>>> http://blog.nanthrax.net >>>> Talend - http://www.talend.com >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >>
