Re: Join streams Apache Spark

2017-05-08 Thread saulshanabrook
I, actually, just ran it in a Docker image. But the point is, it doesn't need to run in the JVM, because it just runs as a separate process. Then your Java (or any other client) code sends messages to it over TCP and it relays them to Spark. On Mon, May 8, 2017 at 4:07 AM tencas [via Apache Spark

Re: Join streams Apache Spark

2017-05-07 Thread saulshanabrook
The script I wrote in Go? No I do not, but it's very easy to compile it to whatever platform you are running on! Doesn't need to be integrated in the same language as the rest of your code. On Sat, May 6, 2017 at 3:13 PM tencas [via Apache Spark User List] < ml+s1001560n28658...@n3.nabble.com> wro

Re: Join streams Apache Spark

2017-05-06 Thread saulshanabrook
Would love to hear if you try it out. I was also considering that. I recently changed to using the file based streaming input. I made another Go script <https://github.com/saulshanabrook/ici.recorder/blob/fd8110e490691cc9e98dce1fefbddba973c29deb/server/main.go> that let's me connect o

Re: Join streams Apache Spark

2017-05-06 Thread saulshanabrook
I wrote a server in Go <https://gist.github.com/saulshanabrook/ce65baf5d82460b655e1232b9fe796d3> that allows many TCP connections for incoming data on one port, writing each line to the client listening on another port. The environmental variable set's what port client's shoul