Hi Lahiru, Good initiative. It seems like we need to use hadoop-hdfs and hadoop common with version 2.8.3. Try adding these dependencies, and exclude the old dependency from the gora, or otherwise, we can do a version upgrade in gora for Hadoop. at the moment we use Hadoop version 2.5.2 in gora but I prefer to upgrade the Hadoop versions in the gora to 2.8.3.
Recently hazelcast developers released 3.0 version, We can try that also. Thanks, Madhawa On Sat, Apr 6, 2019 at 9:53 PM Lahiru Jayasekera <mlpjayasek...@gmail.com> wrote: > Hi all, > I have come up with a poc for hazelcast jet execution engine support for > gora. > > https://github.com/LahiruJayasekara/gora/blob/poc-hazelcast-jet/gora-tutorial/src/main/java/org/apache/gora/tutorial/log/HazelcastJetPOC.java > > Here I have wrote a custom source for jet. What this does is read the > AccessLog table created via LogManager example, and feed the PageView > objects to jet. > > I have also looked into using hdfs connectors of jet. But I encountered a > problem with conflicting hadoop versions. InputFormat class used in gora is > from pacjage 'org.apache.hadoop.mapreduce', but in jet it is ' > org.apache.hadoop.mapred.InputFormat'. > > Anyway the proposed poc works as expected. Please add your suggestions > here. > > Thanks and regards > > -- > Lahiru Jayasekara > Batch 15 > Faculty of Information Technology > University of Moratuwa > 0716492170 >
