Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-168676151
@srowen thanks for the reminder, I just fixed the check style failures,
please help to trigger the test build.
---
If your project is set up for it, you can reply
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48684056
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -51,29 +51,44 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48684054
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -51,29 +51,44 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-168287331
@srowen please help to review latest code changes. I have tested different
cases.(Just setup the entire environment, it is my first PR, thanks for the
patient
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48676000
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -52,46 +52,60 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167957676
Testing in the yarn cluster with new changes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48588447
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -52,46 +52,60 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48588257
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -52,46 +52,60 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167802563
Thanks @srowen, look like previous receive() method included many logic
inside (create socket, release socket, restart in case non-fatch exception), I
almost missed
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167760455
@srowen, thanks for the review. I combined the the resource release logic
from finally block and onStop(), and use synchronized block to avoid possible
release
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48530083
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -52,34 +52,40 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on a diff in the pull request:
https://github.com/apache/spark/pull/10464#discussion_r48527970
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala
---
@@ -52,34 +52,40 @@ class SocketReceiver[T: ClassTag
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167714955
@zsxwing However, it looks unlikely since it sleeps 2 seconds before
calling startReceiver().
Is it possible about thread pool capability issue and 2 seconds
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167711102
@srowen, thanks for the reply. I tested it in my yarn environment, it looks
more stable now.
In current implementation, restart will start new socket in new
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/10464#issuecomment-167038464
Not sure why 2 commits in this PR? Do I need t re-submit this PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
GitHub user guoxu1231 opened a pull request:
https://github.com/apache/spark/pull/10464
[SPARK-12513] [Streaming] SocketReceiver hang in Netcat example
Close client side socket connection before restart socket receiver.
You can merge this pull request into a Git repository by
Github user guoxu1231 commented on the pull request:
https://github.com/apache/spark/pull/8934#issuecomment-162417597
BTW, There is a JIRA to suggest user to prefer using c['column'] instead of
c.column.
@davies, we encountered similar issue, could you paste the J
17 matches
Mail list logo
