Re: Is there something like HiveIO planned?

2016-11-24 Thread Tim Taschke
Hi Jean-Baptiste, sounds great! Yes, please do so. I will check out the contribution guide and join the dev mailing list. Looking forward to work with you on this! Regards, Tim On Fri, Nov 25, 2016 at 3:50 PM, Jean-Baptiste Onofré wrote: > Hi Tim, > > we love contribution

Re: Is there something like HiveIO planned?

2016-11-24 Thread Jean-Baptiste Onofré
Hi Tim, we love contribution ! Are you interested to work with us on HiveIO and HBaseIO ? I can push a skeleton branch and add you as contributor to prepare a PR. WDYT ? Regards JB On 11/25/2016 07:47 AM, Tim Taschke wrote: Great to hear that this is an upcoming feature! Ok, understood

Re: Is there something like HiveIO planned?

2016-11-24 Thread Tim Taschke
Great to hear that this is an upcoming feature! Ok, understood about the Jira creation timing. Btw, I only started playing around with Beam, but if there is any way I could help, I would love to contribute to this. Regards, Tim On Fri, Nov 25, 2016 at 3:31 PM, Jean-Baptiste Onofré

Re: Is there something like HiveIO planned?

2016-11-24 Thread Jean-Baptiste Onofré
Hi Tim, We have a HBaseIO and HiveIO in mind. Ismaël and I discussed about that. It's not yet started, but should be very soon. By the way, I create the Jira only when work started. If you want, we can create HiveIO and HBaseIO related Jira. Regards JB On 11/25/2016 04:34 AM, Tim Taschke

Is there something like HiveIO planned?

2016-11-24 Thread Tim Taschke
Hi, I was wondering, if there is any specific support for Hive through a HiveIO connector planned? I checked the repository, JIRA issues and mailing lists, but could not find anything refering to Hive, which made me wonder why this is. Do people normally use JdbcIO for this or read the files

Re: java.lang.IllegalAccessError with KafkaIO

2016-11-24 Thread Stephan Ewen
I think have seen these kind of errors before. It is the JVM checking upon class loading (lazy linking) that a private/protected access is valid. It could be that this violation is because there are multiple versions of that class loaded - some in the SystemClassLoader and some in the user

java.lang.IllegalAccessError with KafkaIO

2016-11-24 Thread Wayne Collins
Hi, I have run into an issue with launching Beam applications that use KafkaIO on a flink cluster: "java.lang.IllegalAccessError: tried to access method com.google.common.base.Optional.()V from class com.google.common.base.Absent" (full output, and pom.xml below) Other Beam applications

Re: GroupByKey and CombineFn: internals

2016-11-24 Thread Matthias Baetens
Hi Robert, Thanks for brining clarity about the SerializableFunction. The full_list is an argument I give to the function - but doesn't matter a great deal in this discussion... I will try the Sets approach though! Thanks for all the input, got some valuable things to work with :) Matthias On

Re: 回复:how to use key-value storage like redis with PCollection?

2016-11-24 Thread Jean-Baptiste Onofré
Hi Ya-Feng, The RedisIO is on my private repo (not yet public). I will push on public repo asap. Regards JB On 11/24/2016 10:04 AM, 郭亚峰(默岭) wrote: Hi Jean-Baptiste, Morning. I have quite similiar with Jing's case. I wanna join some relatively static data from HBase (which were bulk loaded

回复:how to use key-value storage like redis with PCollection?

2016-11-24 Thread 郭亚峰(默岭)
Hi Jean-Baptiste, Morning.I have quite similiar with Jing's case. I wanna join some relatively static data from HBase (which were bulk loaded everyday) in an unbounded pipeline. I'd like take a look at your code for a reference. I checked your github but couldn't found anything close to RedisIO