Hello and thank you for your reply, I want to run benchmarks from TPC such like this http://tpc.org/tpcds/default5.asp which consist of simple SQL queries such as UPDATE, DELETE and JOIN in order to measure Wayang's performance when connected to Spark, Postgres and GraphChi. Could you please provide me with more details about how to connect those three systems to Wayang as well as Spark, Postgres and GraphChi plugins, especially when they run on different VMs? Excuse me if I misunderstood something, I am a student with no prior experience in these systems. Thank you for your time. Best regards, Lefteris
On Tue, Jan 25, 2022 at 10:35 PM Bertty Contreras <[email protected]> wrote: > Hi Lefteris, > > It is possible to run the query that you describe using Apache Wayang; one > example that could help you understand how to connect different platforms > in one task is the Query3Hybrid [1]. Nevertheless, if you could explain > more about the shape of the query will look, we can give you more hints on > how you could do it. > > I mean with a shape something like the following example: > > The need to run an SQL query in top Postgres, then join the result with > data stored in HDFS or S3 and transform it to execute a page rank using > Graphchi; after the results are processed, the information needs to be > joined with another dataset. > > Let me know the shape, and we will be able provide more information. > > Thank you so much for your question; this will help us a lot to know what > to write in the documentation in the example section :D > > Best regards, > Bertty > > [1] > > https://github.com/apache/incubator-wayang/blob/main/wayang-benchmark/code/main/scala/org/apache/wayang/apps/tpch/queries/Query3Hybrid.scala > > On Tue, Jan 25, 2022 at 4:08 PM Lefteris Lymperopoulos < > [email protected]> wrote: > > > Hello Dev Team, > > Excuse me if this mail isn't appropriate for this mail address. Since I > > could not find any documentation for Wayang besides that in the official > > website I would like to ask you if you could help me with this issue. I > > have 3 VMs that connect to each other. In the first VM I want to run > Spark, > > in the second I want to run Postgres and in the third GraphChi. I also > have > > Wayang installed in the first VM and I intend to develop my app in Java. > Is > > it possible to connect Wayang to Spark, Postgres and GraphChi in order to > > run my queries? If yes, could you please show me how to do it? Or do > these > > platforms have to be in the same VM? Any help would be greatly > appreciated. > > Best regards, > > Lefteris Lymperopoulos > > >
