Hello, I am trying to do this, but it can not find the python packages we are importing.
I tried to start setup python api per this guide: https://github.com/apache/incubator-wayang/tree/9882b860ce1cda66382e9c1fa91e4e657397a233/python But i encountered an error when trying to run python3 -m pip install dist/pywy-1.0.0.tar.gz Do you think this could be related to not being able to find the modules that i try to import in python? ________________________________ From: AurĂ©lien Bertrand <aurelien9.bertr...@gmail.com> Sent: Wednesday, March 26, 2025 1:00 PM To: dev@wayang.apache.org <dev@wayang.apache.org> Subject: Re: Wayang REST api [You don't often get email from aurelien9.bertr...@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Christoffer, You can try to run the following once you have the Python package set up (I recommend creating a conda env specifically for this, and running the file outside the Wayang project): from pywy.dataquanta import WayangContextfrom pywy.platforms.java import JavaPlugin def word_count(): ctx = WayangContext() \ .register({JavaPlugin}) \ .textfile("file://path_to_some_input_text_file.txt") \ .store_textfile("file://path_to_where_you_want_to_store.txt") if __name__ == "__main__": word_count Then you should see in your terminal the JSON plan printed. Of course, you can make it more complex by using other operators đŸ˜‰ Best regards, AurĂ©lien On Wed, 26 Mar 2025 at 12:36, Christoffer Emil Kristensen <c...@itu.dk.invalid> wrote: > Hello Juri, > > We are trying to setup python by the following guide: > https://github.com/apache/incubator-wayang/tree/9882b860ce1cda66382e9c1fa91e4e657397a233/python > > But both me and @Jeppe Nordvi Halvorsen <h...@itu.dk> get this error when > running: > > python3 -m pip install dist/pywy-1.0.0.tar.gz > > > Do you know if there are any prerequisites to start using the python api? > > Also, if you or anyone else have an example of a working wayang plan in > json format that you would be willing to send it would be appreciated. > > > ------------------------------ > *From:* Christoffer Emil Kristensen <c...@itu.dk.INVALID> > *Sent:* Wednesday, March 26, 2025 12:14 PM > *To:* dev@wayang.apache.org <dev@wayang.apache.org> > *Subject:* Re: Wayang REST api > > [You don't often get email from c...@itu.dk.invalid. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > Hello Juri, > > Yes i am aware that the filepath is set to mike//something, but i would > assume that it would fail with something similar to "file not found" and > not a serialization or null pointer error. So, it was just to test that if > it would actually work with the wayang plan and hit the REST endpoint and > start executing a wayang plan. > > I will look into the python api and see if I can get it running. > > Thank you for your reply! > > Best regards, > > Christoffer Kristensen > ________________________________ > From: Juri Petersen <j...@itu.dk.INVALID> > Sent: Tuesday, March 25, 2025 9:06 AM > To: dev@wayang.apache.org <dev@wayang.apache.org> > Subject: Re: Wayang REST api > > [You don't often get email from j...@itu.dk.invalid. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > Hi Christoffer, > What you are observing is not really a bug, but an old example of a > test-plan that we should remove. > If you look at the plans contents, you can see that some data is hardcoded > to be on a system with a mike user, and I am pretty sure this will never be > true for your setup ?? > > I am not sure if any of the example plans in the resources folder there > are up-to-date and work. > It is probably easiest to run one of our python-api examples and just put > a print statement for the json-payload on the python side of things. > Looking at this code, if you execute any of the python examples, a > json-body should be printed: > > > https://github.com/apache/incubator-wayang/blob/9882b860ce1cda66382e9c1fa91e4e657397a233/python/src/pywy/core/core.py#L139 > > I hope this helps, if not let me know. > > Best, > Juri > ________________________________ > From: Alexander Alten <a...@scalytics.io> > Sent: 24 March 2025 17:26 > To: dev@wayang.apache.org <dev@wayang.apache.org> > Subject: Re: Wayang REST api > > Hi Christopher, > > That looks like a bug, would you please open a report for it? > > Thank you, > —Alex > > > On Mar 23, 2025, at 21:52, Christoffer Emil Kristensen > <c...@itu.dk.invalid> wrote: > > > > Hello everyone, > > > > We are two students from the IT-University at Copenhagen working on > creating a web interface for Apache Wayang as part of our master thesis. > > > > We are currently working with the REST api to execute a wayang plan. I > have a few questions though. > > > > When i try to run the plan which is an example in the repo: > https://github.com/apache/incubator-wayang/blob/main/wayang-api/wayang-api-json/src/test/resources/test-plan-a.json > < > https://www.google.com/url?q=https://github.com/apache/incubator-wayang/blob/main/wayang-api/wayang-api-json/src/test/resources/test-plan-a.json&source=gmail-imap&ust=1743367966000000&usg=AOvVaw1SR8clzuiWewqIRRClZGwr > > > > > > I get the following error: > > > > > > > > > > I submit the pla through postman at the following url: > http://localhost:8080/wayang-api-json/submit-plan/json < > https://www.google.com/url?q=http://localhost:8080/wayang-api-json/submit-plan/json&source=gmail-imap&ust=1743367966000000&usg=AOvVaw0cpYF3QFzZlBJcWdHIwXsK > > > > > > With the JSON test-plan-a. And my header only contain Content-Type > application/json > > > > If i for example, try to run another example without the context > specified in the wayang plan i get a deserialization error. Does anyone > have any documentation on how to use the API and if i am missing something. > > > > Btw i would assume that running the plan gave me an error as i might not > have access to the specified filepath, but i would assume that if this was > the problem i would receive another error. > > > > This is the entire error: > > java.lang.NullPointerException > > at > org.apache.wayang.api.json.builder.JsonPlanBuilder.setConfiguration(JsonPlanBuilder.scala:107) > > at > org.apache.wayang.api.json.builder.JsonPlanBuilder.fromPlan(JsonPlanBuilder.scala:82) > > at > org.apache.wayang.api.json.Main$.$anonfun$jsonRoute$7(Main.scala:57) > > at > zio.ZIOCompanionVersionSpecific.$anonfun$attempt$1(ZIOCompanionVersionSpecific.scala:100) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:904) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024) > > at > zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381) > > at zio.internal.FiberRuntime.start(FiberRuntime.scala:1339) > > at zio.Runtime$UnsafeAPIV1.runOrFork(Runtime.scala:160) > > at zio.http.netty.NettyRuntime.run(NettyRuntime.scala:52) > > at zio.http.netty.NettyRuntime.run$(NettyRuntime.scala:29) > > at > zio.http.netty.NettyRuntime$$anon$1.run(NettyRuntime.scala:105) > > at > zio.http.netty.server.ServerInboundHandler.writeResponse(ServerInboundHandler.scala:300) > > at > zio.http.netty.server.ServerInboundHandler.channelRead0(ServerInboundHandler.scala:96) > > at > zio.http.netty.server.ServerInboundHandler.channelRead0(ServerInboundHandler.scala:42) > > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) > > at > io.netty.handler.codec.http.HttpServerKeepAliveHandler.channelRead(HttpServerKeepAliveHandler.java:64) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) > > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) > > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) > > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) > > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) > > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > And when running without the context, as per the following example: > https://github.com/apache/incubator-wayang/blob/main/wayang-api/wayang-api-json/src/test/resources/test-plan-a.json > < > https://www.google.com/url?q=https://github.com/apache/incubator-wayang/blob/main/wayang-api/wayang-api-json/src/test/resources/test-plan-a.json&source=gmail-imap&ust=1743367966000000&usg=AOvVaw1SR8clzuiWewqIRRClZGwr > > > > > > I get the error: > > > > com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot > deserialize value of type > `org.apache.wayang.api.json.operatorfromjson.PlanFromJson` from Array value > (token `JsonToken.START_ARRAY`) > > at [Source: (String)"[ > > { > > "id": 1, > > "cat": "input", > > "input": [], > > "output": [2], > > "operatorName": "textFileInput", > > "data": { > > "filename": "file:///home/mike/json-api-examples/in1.txt" > > } > > }, > > { > > "id": 2, > > "cat": "unary", > > "input": [1], > > "output": [3], > > "operatorName": "map", > > "data": { > > "udf": "(s: String) => s.toInt" > > } > > }, > > { > > "cat": "composed", > > "operatorName": "composed", > > "operators": [ > > { > > "id": 3, > > "cat": "unary", > > "[truncated 716 chars]; line: 1, column: 1] > > at > com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) > > at > com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1752) > > at > com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1526) > > at > com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1473) > > at > com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeFromArray(BeanDeserializer.java:656) > > at > com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:211) > > at > com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187) > > at > com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) > > at > com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825) > > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3772) > > at > com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3755) > > at > org.apache.wayang.api.json.parserutil.ParsePlanFromJson$.parsePlanFromString(ParsePlanFromJson.scala:38) > > at > org.apache.wayang.api.json.Main$.$anonfun$jsonRoute$5(Main.scala:56) > > at scala.util.Try$.apply(Try.scala:213) > > at > org.apache.wayang.api.json.Main$.$anonfun$jsonRoute$4(Main.scala:56) > > at > zio.ZIOCompanionVersionSpecific.$anonfun$attempt$1(ZIOCompanionVersionSpecific.scala:100) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:904) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:890) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:967) > > at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:1024) > > at > zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381) > > at zio.internal.FiberRuntime.start(FiberRuntime.scala:1339) > > at zio.Runtime$UnsafeAPIV1.runOrFork(Runtime.scala:160) > > at zio.http.netty.NettyRuntime.run(NettyRuntime.scala:52) > > at zio.http.netty.NettyRuntime.run$(NettyRuntime.scala:29) > > at > zio.http.netty.NettyRuntime$$anon$1.run(NettyRuntime.scala:105) > > at > zio.http.netty.server.ServerInboundHandler.writeResponse(ServerInboundHandler.scala:300) > > at > zio.http.netty.server.ServerInboundHandler.channelRead0(ServerInboundHandler.scala:96) > > at > zio.http.netty.server.ServerInboundHandler.channelRead0(ServerInboundHandler.scala:42) > > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) > > at > io.netty.handler.codec.http.HttpServerKeepAliveHandler.channelRead(HttpServerKeepAliveHandler.java:64) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) > > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) > > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) > > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) > > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) > > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) > > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) > > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) > > at > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) > > at > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > > at java.base/java.lang.Thread.run(Thread.java:829) > > org.apache.wayang.api.json.exception.WayangApiJsonException: Can't parse > json plan. > > > > > > > > Best regards, > > > > Christoffer Kristensen > > > -- > *Scalytics Connect* > The foundation for secure, scalable, and transparent > AI. > > > > -- > 3401 N. MIAMI AVE. STE 230 > 33127 Miami, Florida > United States > www.scalytics.io<http://www.scalytics.io> <http://www.scalytics.io> > > -- Please consider the > environment before printing this email -- > > Disclaimer: > The content of this > message is confidential. If you have received it by mistake, please inform > us by an email reply and then delete the message. It is forbidden to copy, > forward, or in any way reveal the contents of this message to anyone. The > integrity and security of this email cannot be guaranteed over the > Internet. Therefore, the sender will not be held liable for any damage > caused by the message. >
