Hi, So I created a new operator MapHackItOperator in the wayang basic modules operator. This functions essentially the same as MapOperator in the same module.
Since wayang need a platform to execute this operator and to test, in the wayang platform java modules I updated the Mapping and add MapHackitMapping and JavaMapHackItOperator , this is the translation part as well as the execution, it works with Java Plugin. The problem is for only SparkPlugin I updated the wayang platform spark modules for the SparkPlugin so I created Mapping, MapHackItMapping and SparkMapHackItOperator that extends to MapHackItOperator with the same code as the SparkMap to translate it into SparkOperator, I think the translation part is working but Wayang don't know how to enumerate the plan according to the exception or don't know where to output the results? correct me if I'm wrong. I hope it doesn't make it more confusing. Best regards, Matthew On Thu, Aug 25, 2022, 20:45 jorge Arnulfo Quiané Ruiz <[email protected]> wrote: > Hello Matthew, > > To better understand the problem, are you trying to copy the > JavaMapOperator into a Spark operator? Or you are trying to copy a > SparkOperator into a new spark operator such as you did for Java? > > Best, > Jorge > > On Thu, 25 Aug 2022 at 5:56 PM MatthewJ Sanyoto <[email protected] > > > wrote: > > > Hi, > > > > I have a problem regarding implementing a new custom operator using > Spark. > > So I was trying to map a new operator from Wayang Basic operator to the > > Spark Platform to execute in Spark. > > I tried using the Java Platform (only Java Plugin) and just copied the > > code(MapOperator, Mapping, JavaMapOperator and MapMapping) but with > > different operators name to test and it worked but not for Spark > Platform. > > (only Spark Plugin) > > > > Maybe someone here could please explain to me why I got this error. > > Caused by: org.apache.wayang.core.api.exception.WayangException: No > > implementations that concatenate out@Alternative[2x > > ~Alternative[[MapHackIt[1->1, id=3c947bc5]]], 1e683a3e] with > > [in@Alternative[2x > > ~Alternative[[LocalCallbackSink[1->0, id=69fb6037]]], 6babf3bf]]. > > > > Maybe I am missing out on something for Spark? > > > > Best Regards, > > Matthew > > >
