Re: POC for hazelcast jet integration

2019-05-27 Thread Lahiru Jayasekera
Sure, will do that. Thanks On Mon, May 27, 2019 at 2:41 PM Kevin Ratnasekera wrote: > Hi Lahiru, > > As per the offline chat we had, please start a new mail thread on adding > design details, related to approach ( writing custom source and sink > connectors [3] ) you have taken on

Re: POC for hazelcast jet integration

2019-05-27 Thread Kevin Ratnasekera
Hi Lahiru, As per the offline chat we had, please start a new mail thread on adding design details, related to approach ( writing custom source and sink connectors [3] ) you have taken on implementation of Jet execution engine support. That way community can comment on your project work. [1]

Re: POC for hazelcast jet integration

2019-05-19 Thread Lahiru Jayasekera
Hi all, This is the[1] hdfs connector for hazelcast jet. Here the problem is hazelcast jet is using `org.apache.hadoop.mapred` package and gora is using `org.apache.hadoop.mapreduce` package. These are two APIs exposed by hadoop to create hadoop jobs and etc. Eventhough these two are similar in

Re: POC for hazelcast jet integration

2019-04-26 Thread Lahiru Jayasekera
Hi Madhawa, Sorry for the late reply. Sure I'll try that and let you know. Thanks On Tue, Apr 23, 2019 at 2:24 PM Madhawa Kasun Gunasekara < madhaw...@gmail.com> wrote: > Hi Lahiru, > > Good initiative. > It seems like we need to use hadoop-hdfs and hadoop common with version > 2.8.3. Try

Re: POC for hazelcast jet integration

2019-04-23 Thread Madhawa Kasun Gunasekara
Hi Lahiru, Good initiative. It seems like we need to use hadoop-hdfs and hadoop common with version 2.8.3. Try adding these dependencies, and exclude the old dependency from the gora, or otherwise, we can do a version upgrade in gora for Hadoop. at the moment we use Hadoop version 2.5.2 in gora

POC for hazelcast jet integration

2019-04-06 Thread Lahiru Jayasekera
Hi all, I have come up with a poc for hazelcast jet execution engine support for gora. https://github.com/LahiruJayasekara/gora/blob/poc-hazelcast-jet/gora-tutorial/src/main/java/org/apache/gora/tutorial/log/HazelcastJetPOC.java Here I have wrote a custom source for jet. What this does is read