hooks priority.
Author: Tathagata Das
Closes #6307 from tdas/SPARK-7776 and squashes the following commits:
e3d5475 [Tathagata Das] Added conf to specify graceful shutdown
4c18652 [Tathagata Das] Added shutdown hook to StreamingContxt.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
hooks priority.
Author: Tathagata Das
Closes #6307 from tdas/SPARK-7776 and squashes the following commits:
e3d5475 [Tathagata Das] Added conf to specify graceful shutdown
4c18652 [Tathagata Das] Added shutdown hook to StreamingContxt.
(cherry picked from commit d68ea24d60ce1aa55b06a8c107f42544
ver (it's
receiverInfo field is a empty map) causing it to be stuck in infinite loop
while waiting for running flag to be set to false.
Author: Tathagata Das
Closes #6348 from tdas/SPARK-7788 and squashes the following commits:
2584683 [Tathagata Das] Added receiver id in thread name
the receiver (it's
receiverInfo field is a empty map) causing it to be stuck in infinite loop
while waiting for running flag to be set to false.
Author: Tathagata Das
Closes #6348 from tdas/SPARK-7788 and squashes the following commits:
2584683 [Tathagata Das] Added receiver id in thread name
Repository: spark
Updated Branches:
refs/heads/branch-1.4 c8eb76ba6 -> ea9db50bc
[SPARK-][Streaming] Handle the case when there is no block in a batch
In the old implementation, if a batch has no block,
`areWALRecordHandlesPresent` will be `true` and it will return
`WriteAheadLogBackedBl
Repository: spark
Updated Branches:
refs/heads/master a40bca011 -> ad0badba1
[SPARK-][Streaming] Handle the case when there is no block in a batch
In the old implementation, if a batch has no block,
`areWALRecordHandlesPresent` will be `true` and it will return
`WriteAheadLogBackedBlockR
Repository: spark
Updated Branches:
refs/heads/master e7c7e51f2 -> b7ab0299b
[SPARK-7497] [PYSPARK] [STREAMING] fix streaming flaky tests
Increase the duration and timeout in streaming python tests.
Author: Davies Liu
Closes #6239 from davies/flaky_tests and squashes the following commits:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 2f41cf3e2 -> d023300f4
[SPARK-7497] [PYSPARK] [STREAMING] fix streaming flaky tests
Increase the duration and timeout in streaming python tests.
Author: Davies Liu
Closes #6239 from davies/flaky_tests and squashes the following commi
IVE.
The solution in this PR is to stop the internal scheduler if start throw
exception, and mark the context as STOPPED.
Author: Tathagata Das
Closes #6559 from tdas/SPARK-7958 and squashes the following commits:
20b2ec1 [Tathagata Das] Added synchronized
790b617 [Tathagata Das] Handled except
IVE.
The solution in this PR is to stop the internal scheduler if start throw
exception, and mark the context as STOPPED.
Author: Tathagata Das
Closes #6559 from tdas/SPARK-7958 and squashes the following commits:
20b2ec1 [Tathagata Das] Added synchronized
790b617 [Tathagata Das] Handled exception
Repository: spark
Updated Branches:
refs/heads/branch-1.4 f71a09de6 -> fa292dc3d
[SPARK-8015] [FLUME] Remove Guava dependency from flume-sink.
The minimal change would be to disable shading of Guava in the module,
and rely on the transitive dependency from other libraries instead. But
since Gu
Repository: spark
Updated Branches:
refs/heads/master 1bb5d716c -> 0071bd8d3
[SPARK-8015] [FLUME] Remove Guava dependency from flume-sink.
The minimal change would be to disable shading of Guava in the module,
and rely on the transitive dependency from other libraries instead. But
since Guava'
emote-tracking branch 'asf/master'
3572180 [Hari Shreedharan] Adding a license header, making Jenkins happy.
799509f [Hari Shreedharan] Fix a compile issue.
3c5194c [Hari Shreedharan] Merge remote-tracking branch 'asf/master'
d248d22 [harishreedharan] Merge pull request #1 from tdas/fl
Repository: spark
Updated Branches:
refs/heads/master 284771efb -> a32f0fb73
[SPARK-2103][Streaming] Change to ClassTag for KafkaInputDStream and fix
reflection issue
This PR updates previous Manifest for KafkaInputDStream's Decoder to ClassTag,
also fix the problem addressed in
[SPARK-2103
Repository: spark
Updated Branches:
refs/heads/master f5d9bea20 -> b270309d7
[SPARK-2379] Fix the bug that streaming's receiver may fall into a dead loop
Author: joyyoj
Closes #1694 from joyyoj/SPARK-2379 and squashes the following commits:
d73790d [joyyoj] SPARK-2379 Fix the bug that strea
Repository: spark
Updated Branches:
refs/heads/branch-1.0 886508d3b -> 952e0d698
[SPARK-2379] Fix the bug that streaming's receiver may fall into a dead loop
Author: joyyoj
Closes #1694 from joyyoj/SPARK-2379 and squashes the following commits:
d73790d [joyyoj] SPARK-2379 Fix the bug that s
Repository: spark
Updated Branches:
refs/heads/master 67bd8e3c2 -> 91f9504e6
http://git-wip-us.apache.org/repos/asf/spark/blob/91f9504e/extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisReceiverSuite.scala
---
[SPARK-1981] Add AWS Kinesis streaming support
Author: Chris Fregly
Closes #1434 from cfregly/master and squashes the following commits:
4774581 [Chris Fregly] updated docs, renamed retry to retryRandom to be more
clear, removed retries around store() method
0393795 [Chris Fregly] moved Kinesi
[SPARK-1981] Add AWS Kinesis streaming support
Author: Chris Fregly
Closes #1434 from cfregly/master and squashes the following commits:
4774581 [Chris Fregly] updated docs, renamed retry to retryRandom to be more
clear, removed retries around store() method
0393795 [Chris Fregly] moved Kinesi
Repository: spark
Updated Branches:
refs/heads/branch-1.1 91de0dc16 -> bb0ac6d7c
http://git-wip-us.apache.org/repos/asf/spark/blob/bb0ac6d7/extras/kinesis-asl/src/test/scala/org/apache/spark/streaming/kinesis/KinesisReceiverSuite.scala
---
Author: tdas
Date: Tue Aug 5 00:19:19 2014
New Revision: 6045
Log:
Adding spark-1.0.2-rc1
Added:
dev/spark/spark-1.0.2-rc1/
dev/spark/spark-1.0.2-rc1/spark-1.0.2-bin-cdh4.tgz (with props)
dev/spark/spark-1.0.2-rc1/spark-1.0.2-bin-cdh4.tgz.asc
dev/spark/spark-1.0.2-rc1/spark
Repository: spark
Updated Tags: refs/tags/v0.7.0-bizo-1 [created] 02b4a0d96
refs/tags/v0.9.1-rc1 [created] 20a2d247a
refs/tags/v0.9.1-rc2 [created] 95d70498b
refs/tags/v0.9.1-rc3 [created] 931ac8a0e
refs/tags/v1.0.0-rc1 [created] c8d0eb980
refs/tags/v1.0.0-rc10 [created] eafc6f6f4
refs
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc2 [deleted] 327ab1e24
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc7 [deleted] bc81ecfdc
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc6 [deleted] aab03f5f9
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc5 [deleted] 04ce12153
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc8 [deleted] 9abe0192a
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc3 [deleted] e7c46933a
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc9 [deleted] 093aeb40f
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.1-rc1 [deleted] 2a9114542
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc10 [deleted] eafc6f6f4
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.1-rc2 [deleted] 9c4493512
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v0.9.2-rc1 [deleted] 5a3020615
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v0.9.1-rc3 [deleted] 931ac8a0e
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v0.9.1-rc1 [deleted] 20a2d247a
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v0.9.1-rc2 [deleted] 95d70498b
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.0-rc1 [deleted] c8d0eb980
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Tags: refs/tags/v1.0.2-rc1 [deleted] 913e3de6d
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.0 439cdc581 -> 3fbe3447e
Updating version for Spark 1.0.3 development.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3fbe3447
Tree: http://git-wip-us.apache.org/rep
Author: tdas
Date: Tue Aug 5 01:53:43 2014
New Revision: 6047
Log:
Moving Spark 1.0.2 from dev to release.
Added:
release/spark/spark-1.0.2/
- copied from r6046, dev/spark/spark-1.0.2-rc1/
Removed:
dev/spark/spark-1.0.2-rc1
ues.
@tdas, would you mind reviewing this PR? Thanks a lot.
Author: jerryshao
Closes #1751 from jerryshao/kafka-unit-test and squashes the following commits:
b6a505f [jerryshao] code refactor according to comments
5222330 [jerryshao] Change JavaKafkaStreamSuite to better test it
5525f10 [jerrys
aky
issues.
@tdas, would you mind reviewing this PR? Thanks a lot.
Author: jerryshao
Closes #1751 from jerryshao/kafka-unit-test and squashes the following commits:
b6a505f [jerryshao] code refactor according to comments
5222330 [jerryshao] Change JavaKafkaStreamSuite to better test it
5525
Author: tdas
Date: Tue Aug 5 23:40:54 2014
New Revision: 1616049
URL: http://svn.apache.org/r1616049
Log:
Adding Spark 1.0.2
[This commit notification would consist of 386 parts,
which exceeds the limit of 50 ones, so it was shortened to the summary
Author: tdas
Date: Tue Aug 5 23:43:56 2014
New Revision: 1616050
URL: http://svn.apache.org/r1616050
Log:
Small update to Spark 1.0.2.
Modified:
spark/news/_posts/2014-08-05-spark-1-0-2-released.md
spark/site/news/index.html
spark/site/news/spark-1-0-2-released.html
Modified: spark
Repository: spark
Updated Branches:
refs/heads/master e537b33c6 -> c6889d2cb
[HOTFIX][Streaming] Handle port collisions in flume polling test
This is failing my tests in #1777. @tdas
Author: Andrew Or
Closes #1803 from andrewor14/fix-flaky-streaming-test and squashes the
following comm
Repository: spark
Updated Branches:
refs/heads/master bfa09b01d -> 99243288b
[SPARK-1981] updated streaming-kinesis.md
fixed markup, separated out sections more-clearly, more thorough explanations
Author: Chris Fregly
Closes #1757 from cfregly/master and squashes the following commits:
9b1
Repository: spark
Updated Branches:
refs/heads/branch-1.1 4f776dfab -> 826356725
[SPARK-1981] updated streaming-kinesis.md
fixed markup, separated out sections more-clearly, more thorough explanations
Author: Chris Fregly
Closes #1757 from cfregly/master and squashes the following commits:
Repository: spark
Updated Branches:
refs/heads/master 99243288b -> 95470a03a
[HOTFIX][STREAMING] Allow the JVM/Netty to decide which port to bind to in
Flume Polling Tests.
Author: Hari Shreedharan
Closes #1820 from harishreedharan/use-free-ports and squashes the following
commits:
b93906
rom ``streaming``
- Rewrote accuracy and convergence tests to use ``setupStreams`` and
``runStreams``
- Added new test for the accuracy of predictions generated by ``predictOnValue``
These tests should run faster, be easier to extend/maintain, and provide a
reference for new tests.
mengxr tdas
Aut
rom ``streaming``
- Rewrote accuracy and convergence tests to use ``setupStreams`` and
``runStreams``
- Added new test for the accuracy of predictions generated by ``predictOnValue``
These tests should run faster, be easier to extend/maintain, and provide a
reference for new tests.
mengxr t
Repository: spark
Updated Branches:
refs/heads/master 0a7ef6339 -> 8c5a22269
[SPARK-3054][STREAMING] Add unit tests for Spark Sink.
This patch adds unit tests for Spark Sink.
It also removes the private[flume] for Spark Sink,
since the sink is instantiated from Flume configuration (looks like
Repository: spark
Updated Branches:
refs/heads/branch-1.1 5b22ebf68 -> 9b2909955
[SPARK-3054][STREAMING] Add unit tests for Spark Sink.
This patch adds unit tests for Spark Sink.
It also removes the private[flume] for Spark Sink,
since the sink is instantiated from Flume configuration (looks
[HOTFIX][STREAMING] Allow the JVM/Netty to decide which port to bind to in
Flume Polling Tests.
Author: Hari Shreedharan
Closes #1820 from harishreedharan/use-free-ports and squashes the following
commits:
b939067 [Hari Shreedharan] Remove unused import.
67856a8 [Hari Shreedharan] Remove find
Repository: spark
Updated Branches:
refs/heads/branch-1.1 eba399b3c -> 44856654c
[HOTFIX][Streaming] Handle port collisions in flume polling test
This is failing my tests in #1777. @tdas
Author: Andrew Or
Closes #1803 from andrewor14/fix-flaky-streaming-test and squashes the
follow
Repository: spark
Updated Branches:
refs/heads/master 220f41368 -> cd30db566
SPARK-2798 [BUILD] Correct several small errors in Flume module pom.xml files
(EDIT) Since the scalatest issue was since resolved, this is now about a few
small problems in the Flume Sink `pom.xml`
- `scalatest` is
Repository: spark
Updated Branches:
refs/heads/branch-1.1 69a17f119 -> ff616fd7b
SPARK-2798 [BUILD] Correct several small errors in Flume module pom.xml files
(EDIT) Since the scalatest issue was since resolved, this is now about a few
small problems in the Flume Sink `pom.xml`
- `scalatest`
Repository: spark
Updated Branches:
refs/heads/branch-1.1 7286d5707 -> 1d468df33
[SPARK-3154][STREAMING] Make FlumePollingInputDStream shutdown cleaner.
Currently lot of errors get thrown from Avro IPC layer when the dstream
or sink is shutdown. This PR cleans it up. Some refactoring is done i
Repository: spark
Updated Branches:
refs/heads/master 171a41cb0 -> 6f671d04f
[SPARK-3154][STREAMING] Make FlumePollingInputDStream shutdown cleaner.
Currently lot of errors get thrown from Avro IPC layer when the dstream
or sink is shutdown. This PR cleans it up. Some refactoring is done in th
Repository: spark
Updated Branches:
refs/heads/master aa7de128c -> e9bb12bea
[SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis
- Include kinesis in the unidocs
- Hide non-public classes from docs
Author: Tathagata Das
Closes #2239 from tdas/kinesis-doc-fix and squashes
Repository: spark
Updated Branches:
refs/heads/branch-1.1 9b0cff2d4 -> 0c8183cb3
[SPARK-1981][Streaming][Hotfix] Fixed docs related to kinesis
- Include kinesis in the unidocs
- Hide non-public classes from docs
Author: Tathagata Das
Closes #2239 from tdas/kinesis-doc-fix and squashes
Das
Author: Jacek Laskowski
Closes #2254 from tdas/streaming-doc-fix and squashes the following commits:
e45c6d7 [Jacek Laskowski] More fixes from an old PR
5125316 [Tathagata Das] Fixed links
dc02f26 [Tathagata Das] Refactored streaming kinesis guide and made many other
changes.
acbc3e3 [Tathagata
hor: Jacek Laskowski
Closes #2254 from tdas/streaming-doc-fix and squashes the following commits:
e45c6d7 [Jacek Laskowski] More fixes from an old PR
5125316 [Tathagata Das] Fixed links
dc02f26 [Tathagata Das] Refactored streaming kinesis guide and made many other
changes.
acbc3e3 [Tathagata
hor: Tathagata Das
Author: Chris Fregly
Closes #2307 from tdas/streaming-doc-fix1 and squashes the following commits:
ec40b5d [Tathagata Das] Updated figure with kinesis
fdb9c5e [Tathagata Das] Fixed style issues with kinesis guide
036d219 [Chris Fregly] updated kinesis docs and added an arch diag
hor: Tathagata Das
Author: Chris Fregly
Closes #2307 from tdas/streaming-doc-fix1 and squashes the following commits:
ec40b5d [Tathagata Das] Updated figure with kinesis
fdb9c5e [Tathagata Das] Fixed style issues with kinesis guide
036d219 [Chris Fregly] updated kinesis docs and added an arch diag
Repository: spark
Updated Branches:
refs/heads/master b8487713d -> c3f2a8588
SPARK-2932 [STREAMING] Move MasterFailureTest out of "main" source directory
(HT @vanzin) Whatever the reason was for having this test class in `main`, if
there is one, appear to be moot. This may have been a result
Repository: spark
Updated Branches:
refs/heads/master d3a3840e0 -> 8764fe368
SPARK-3744 [STREAMING] FlumeStreamSuite will fail during port contention
Since it looked quite easy, I took the liberty of making a quick PR that just
uses `Utils.startServiceOnPort` to fix this. It works locally for
Repository: spark
Updated Branches:
refs/heads/branch-1.1 b4f690d36 -> 24ee61625
Typo error in KafkaWordCount example
topicpMap to topicMap
Author: Gaspar Munoz
Closes #2614 from gasparms/patch-1 and squashes the following commits:
00aab2c [Gaspar Munoz] Typo error in KafkaWordCount exampl
Repository: spark
Updated Branches:
refs/heads/master 8cc70e7e1 -> b81ee0b46
Typo error in KafkaWordCount example
topicpMap to topicMap
Author: Gaspar Munoz
Closes #2614 from gasparms/patch-1 and squashes the following commits:
00aab2c [Gaspar Munoz] Typo error in KafkaWordCount example
ore changes.
Author: Tathagata Das
Closes #4384 from tdas/new-kafka-fixes and squashes the following commits:
7c931c3 [Tathagata Das] Small update
3ed9284 [Tathagata Das] updated scala doc
83d0402 [Tathagata Das] Added JavaDirectKafkaWordCount example.
26df23c [Tathagata Das] Updates based on
ore changes.
Author: Tathagata Das
Closes #4384 from tdas/new-kafka-fixes and squashes the following commits:
7c931c3 [Tathagata Das] Small update
3ed9284 [Tathagata Das] updated scala doc
83d0402 [Tathagata Das] Added JavaDirectKafkaWordCount example.
26df23c [Tathagata Das] Updates based on
Repository: spark
Updated Branches:
refs/heads/master c2131c0cd -> 658687b25
[SPARK-4964] [Streaming] refactor createRDD to take leaders via map instead of
array
Author: cody koeninger
Closes #4511 from koeninger/kafkaRdd-leader-to-broker and squashes the
following commits:
f7151d4 [cody
Repository: spark
Updated Branches:
refs/heads/branch-1.3 e57c81b8c -> 811d1798d
[SPARK-4964] [Streaming] refactor createRDD to take leaders via map instead of
array
Author: cody koeninger
Closes #4511 from koeninger/kafkaRdd-leader-to-broker and squashes the
following commits:
f7151d4 [c
is not its
intention. I will be testing this patch multiple times to understand its
flakiness.
PS: This was caused by my refactoring in
https://github.com/apache/spark/pull/4384/
koeninger check it out.
Author: Tathagata Das
Closes #4597 from tdas/kafka-flaky-test and squashes the follow
eam. Which is not its
intention. I will be testing this patch multiple times to understand its
flakiness.
PS: This was caused by my refactoring in
https://github.com/apache/spark/pull/4384/
koeninger check it out.
Author: Tathagata Das
Closes #4597 from tdas/kafka-flaky-test and squashes
hor: Tathagata Das
Closes #4753 from tdas/SPARK-5993 and squashes the following commits:
c390db8 [Tathagata Das] Fix assembly jar location of kafka-assembly
(cherry picked from commit 922b43b3cc1cca04e0313bf9e31c5f944ac06d1f)
Signed-off-by: Tathagata Das
Project: http://git-wip-us.apache.org/re
hor: Tathagata Das
Closes #4753 from tdas/SPARK-5993 and squashes the following commits:
c390db8 [Tathagata Das] Fix assembly jar location of kafka-assembly
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/922b43b3
Tree: http://
Repository: spark
Updated Branches:
refs/heads/master d17cb2ba3 -> 5f7f3b938
[Streaming][Minor] Remove useless type signature of Java Kafka direct stream API
cc tdas .
Author: Saisai Shao
Closes #4817 from jerryshao/signature-minor-fix and squashes the following
commits:
eebfaac [Sai
Repository: spark
Updated Branches:
refs/heads/branch-1.3 117e10c18 -> ceebe3c60
[Streaming][Minor] Remove useless type signature of Java Kafka direct stream API
cc tdas .
Author: Saisai Shao
Closes #4817 from jerryshao/signature-minor-fix and squashes the following
commits:
eebf
Repository: spark
Updated Branches:
refs/heads/master 9d6c5aeeb -> 9eb22ece1
[SPARK-6127][Streaming][Docs] Add Kafka to Python api docs
davies
Author: Tathagata Das
Closes #4860 from tdas/SPARK-6127 and squashes the following commits:
82de92a [Tathagata Das] Add Kafka to Python api d
Repository: spark
Updated Branches:
refs/heads/branch-1.3 11389f026 -> ffd059109
[SPARK-6127][Streaming][Docs] Add Kafka to Python api docs
davies
Author: Tathagata Das
Closes #4860 from tdas/SPARK-6127 and squashes the following commits:
82de92a [Tathagata Das] Add Kafka to Python
975 from tdas/streaming-sql-examples and squashes the following
commits:
705cba1 [Tathagata Das] Fixed python lint error
75a3fad [Tathagata Das] Fixed python lint error
5fbf789 [Tathagata Das] Removed empty lines at the end
874b943 [Tathagata Das] Added examples streaming + sql examples.
Proj
ses #4975 from tdas/streaming-sql-examples and squashes the following
commits:
705cba1 [Tathagata Das] Fixed python lint error
75a3fad [Tathagata Das] Fixed python lint error
5fbf789 [Tathagata Das] Removed empty lines at the end
874b943 [Tathagata Das] Added examples streaming + sql examples.
(che
the main streaming guide
- Improved details on the fault-tolerance semantics
Generated docs located here
http://people.apache.org/~tdas/spark-1.3.0-temp-docs/streaming-programming-guide.html#fault-tolerance-semantics
More things to add:
- Configuration for Kafka receive rate
- May be add concurrentJ
the main streaming guide
- Improved details on the fault-tolerance semantics
Generated docs located here
http://people.apache.org/~tdas/spark-1.3.0-temp-docs/streaming-programming-guide.html#fault-tolerance-semantics
More things to add:
- Configuration for Kafka receive rate
- May be
Repository: spark
Updated Branches:
refs/heads/branch-1.3 67fa6d1f8 -> 47cce984e
[SPARK-6077] Remove streaming tab while stopping StreamingContext
Currently we would create a new streaming tab for each streamingContext even if
there's already one on the same sparkContext which would cause dup
Repository: spark
Updated Branches:
refs/heads/master d19efeddc -> f149b8b5e
[SPARK-6077] Remove streaming tab while stopping StreamingContext
Currently we would create a new streaming tab for each streamingContext even if
there's already one on the same sparkContext which would cause duplica
(to maintain existing behavior).
Author: Tathagata Das
Closes #5024 from tdas/SPARK-6331 and squashes the following commits:
392fd44 [Tathagata Das] Fixed naming issue.
c7c0b99 [Tathagata Das] Addressed comments.
6a0857c [Tathagata Das] Updated testsuites.
222485d [Tathagata Das] Load new master
(to maintain existing behavior).
Author: Tathagata Das
Closes #5024 from tdas/SPARK-6331 and squashes the following commits:
392fd44 [Tathagata Das] Fixed naming issue.
c7c0b99 [Tathagata Das] Addressed comments.
6a0857c [Tathagata Das] Updated testsuites.
222485d [Tathagata Das] Load new master URL
Das
Closes #5008 from tdas/SPARK-6222 and squashes the following commits:
7315bc2 [Tathagata Das] Removed empty line.
c438de4 [Tathagata Das] Revert unnecessary change.
5e98374 [Tathagata Das] Added unit test
50cb60b [Tathagata Das] Fixed style issue
295ca5c [Tathagata Das] Fixing SPARK-6
ata Das
Closes #5008 from tdas/SPARK-6222 and squashes the following commits:
7315bc2 [Tathagata Das] Removed empty line.
c438de4 [Tathagata Das] Revert unnecessary change.
5e98374 [Tathagata Das] Added unit test
50cb60b [Tathagata Das] Fixed style issue
295ca5c [Tathagata Das] Fixing SPARK-6
Repository: spark
Updated Branches:
refs/heads/master 2a5b5fd4c -> b004150ad
http://git-wip-us.apache.org/repos/asf/spark/blob/b004150a/docs/streaming-programming-guide.md
--
diff --git a/docs/streaming-programming-guide.md
b/
- Update Receiver guide to discuss reliable vs unreliable receivers.
Author: Tathagata Das
Author: Josh Rosen
Author: Josh Rosen
Closes #3653 from tdas/streaming-doc-update-1.2 and squashes the following
commits:
f53154a [Tathagata Das] Addressed Josh's comments.
ce299e4 [Tathagata Das]
Repository: spark
Updated Branches:
refs/heads/branch-1.2 1eb3ec5f0 -> c3b071383
http://git-wip-us.apache.org/repos/asf/spark/blob/c3b07138/docs/streaming-programming-guide.md
--
diff --git a/docs/streaming-programming-guide.md
- Update Receiver guide to discuss reliable vs unreliable receivers.
Author: Tathagata Das
Author: Josh Rosen
Author: Josh Rosen
Closes #3653 from tdas/streaming-doc-update-1.2 and squashes the following
commits:
f53154a [Tathagata Das] Addressed Josh's comments.
ce299e4 [Tathagata Das]
Repository: spark
Updated Branches:
refs/heads/branch-1.2 6a46cc3c8 -> 01adf45a9
[SPARK-4802] [streaming] Remove receiverInfo once receiver is de-registered
Once the streaming receiver is de-registered at executor, the
`ReceiverTrackerActor` needs to
remove the corresponding reveiverInfo fr
Repository: spark
Updated Branches:
refs/heads/master 96281cd0c -> 10d69e9cb
[SPARK-4802] [streaming] Remove receiverInfo once receiver is de-registered
Once the streaming receiver is de-registered at executor, the
`ReceiverTrackerActor` needs to
remove the corresponding reveiverInfo from t
Repository: spark
Updated Branches:
refs/heads/branch-1.1 3bce43f67 -> b1de461a7
[SPARK-4802] [streaming] Remove receiverInfo once receiver is de-registered
Once the streaming receiver is de-registered at executor, the
`ReceiverTrackerActor` needs to
remove the corresponding reveiverInfo fr
is needless and will
hurt the throughput of streaming application.
Hi tdas , as per discussed about this issue, I fixed with this implementation,
I'm not is this the way you want, would you mind taking a look at it? Thanks a
lot.
Author: jerryshao
Closes #3534 from jerryshao/SPARK-4671 and
is needless and will
hurt the throughput of streaming application.
Hi tdas , as per discussed about this issue, I fixed with this implementation,
I'm not is this the way you want, would you mind taking a look at it? Thanks a
lot.
Author: jerryshao
Closes #3534 from jerryshao/SPARK-
Repository: spark
Updated Branches:
refs/heads/master 29fabb1b5 -> b4d0db80a
[SPARK-4873][Streaming] Use `Future.zip` instead of `Future.flatMap`(for-loop)
in WriteAheadLogBasedBlockHandler
Use `Future.zip` instead of `Future.flatMap`(for-loop). `zip` implies these two
Futures will run concu
401 - 500 of 1024 matches
Mail list logo
