Github user liyezhang556520 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r21142893
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManag
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17646960
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTrackingVector.scala
---
@@ -0,0 +1,46 @@
+/*
+ * Licensed to the Apache Softwar
Github user cloud-fan commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-55850975
Hi @andrewor14 , I think about your fix and focus on the timing to release
unrollMemory. If we unroll a partition successfully, currently we release
unrollMemory immedi
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17645466
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManager) e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17645275
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManager) ex
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17617153
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManager) e
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17584855
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManager) ex
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17584609
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -118,21 +118,29 @@ private[spark] class CacheManager(blockManager:
BlockManager) ex
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17581688
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
maxM
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17576857
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17518799
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
maxM
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17489658
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17489633
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17470235
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
maxM
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r17469980
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
maxM
Github user liyezhang556520 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r16703247
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r16690496
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user liyezhang556520 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r16100690
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,93 @@ private class MemoryStore(blockManager: BlockManager,
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50290568
Great. Thanks for reviewing @pwendell @mateiz @mridulm.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1165
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50289038
Thanks Andrew! The changes look good to me -- I've merged this in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50281752
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50280496
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17245/consoleFull
---
If
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50280386
I have addressed your latest comments and rebased to master. Anything else?
---
If your project is set up for it, you can reply to this email and have your
reply appea
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50246395
@andrewor14 I looked through this and it looks good to me. Made a few very
small comments to clarify the algorithm. Thanks for adding the test with
multiple blocks being u
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15435485
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15435457
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15435454
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +193,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50219496
@mateiz There were a couple of correctness issues with the previous code
that my latest commits have fixed (see commit message for more detail). After
fixing these I h
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50219182
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50217815
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17208/consoleFull
---
If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15413324
--- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala ---
@@ -52,22 +50,21 @@ class CacheManagerSuite extends FunSuite with
BeforeAndAfter
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15412910
--- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala ---
@@ -52,22 +50,21 @@ class CacheManagerSuite extends FunSuite with
BeforeAndAfter
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50109743
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50107921
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17162/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50102333
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15383252
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -275,8 +426,36 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15383242
--- Diff: core/src/test/scala/org/apache/spark/CacheManagerSuite.scala ---
@@ -52,22 +50,21 @@ class CacheManagerSuite extends FunSuite with
BeforeAndAfter wit
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15383171
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -275,8 +426,36 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15383154
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -275,8 +426,36 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15383132
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -20,25 +20,43 @@ package org.apache.spark.storage
import java.nio.ByteBuffe
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50099588
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17155/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50097869
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50096538
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17144/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50074496
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-50069722
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17133/consoleFull
---
If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15363550
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -215,7 +361,10 @@ private class MemoryStore(blockManager: BlockManager,
max
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331347
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -87,9 +97,47 @@ private class MemoryStore(blockManager: BlockManager,
maxMemory
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331245
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +189,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331224
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +189,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49970876
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331171
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +189,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331108
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -215,7 +361,10 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331070
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -215,7 +361,10 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331051
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -225,9 +374,18 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15331032
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +189,104 @@ private class MemoryStore(blockManager: BlockManager,
maxMem
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49969030
@pwendell @mateiz I have updated the unroll logic to do something similar
to EAOM. More details are in the commit message. I have also added some
synchronization logic
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49968995
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17099/consoleFull
---
If
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15271976
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMe
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49831796
BTW for these changes it would be good to perf-test it, e.g. design a
workload where your blocks do fit in memory (just on one multicore machine is
fine) and make sure the
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270521
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -87,9 +97,46 @@ private class MemoryStore(blockManager: BlockManager,
maxMemory
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270516
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -140,14 +148,36 @@ private[spark] class CacheManager(blockManager:
BlockManager) exten
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49831426
Hey @andrewor14 I had some comments on the way this decides when to check
memory usage. It seems quite expensive in the current form.
---
If your project is set up for it
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270480
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270415
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270377
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15270358
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMemo
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49809451
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49797911
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16986/consoleFull
---
If
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49797626
test this please!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fe
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49797163
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49797224
test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this fea
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49795173
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49782460
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16979/consoleFull
---
If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15242324
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -140,14 +145,36 @@ private[spark] class CacheManager(blockManager:
BlockManager) e
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15214991
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTracker.scala ---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Founda
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15214933
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -140,14 +145,36 @@ private[spark] class CacheManager(blockManager:
BlockManager) ext
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15214812
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -124,15 +124,20 @@ private[spark] class CacheManager(blockManager:
BlockManager) ext
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15214670
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -124,15 +124,20 @@ private[spark] class CacheManager(blockManager:
BlockManager) ext
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49685834
I see, we should revisit that, but probably later. IMO if you're going to
store them serialized, you're going to pay for that deserialization cost later
anyway. With the c
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49685219
@mateiz Yes, currently we unroll it in deserialized form even if we only
want to store it in serialized form. One issue with storing it directly in
bytes is that `Cach
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49682053
Hey @andrewor14, one question here just to make sure I understand: if the
data is supposed to be stored as MEMORY_ONLY_SER, will this code still unroll
it in an un-seriali
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15195362
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -561,13 +562,14 @@ private[spark] class BlockManager(
iter
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15195205
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -463,16 +463,15 @@ private[spark] class BlockManager(
v
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15194738
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTrackingAppendOnlyBuffer.scala
---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15194471
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -140,14 +145,36 @@ private[spark] class CacheManager(blockManager:
BlockManager) e
Github user colorant commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15153652
--- Diff: core/src/main/scala/org/apache/spark/storage/MemoryStore.scala ---
@@ -141,6 +188,88 @@ private class MemoryStore(blockManager: BlockManager,
maxMe
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15147430
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTracker.scala ---
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundati
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15147427
--- Diff: core/src/main/scala/org/apache/spark/CacheManager.scala ---
@@ -140,14 +145,36 @@ private[spark] class CacheManager(blockManager:
BlockManager) exten
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15147423
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -463,16 +463,15 @@ private[spark] class BlockManager(
val v
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15147419
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -463,16 +463,15 @@ private[spark] class BlockManager(
val v
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15147416
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTrackingAppendOnlyBuffer.scala
---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1165#discussion_r15145476
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/SizeTrackingAppendOnlyBuffer.scala
---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49516746
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16851/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49497172
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49494682
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16840/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49491986
QA results for PR 1165:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49485183
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16836/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49475344
QA results for PR 1165:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds the following public classes
(experimental):case class Sample(size: Lon
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1165#issuecomment-49465699
QA tests have started for PR 1165. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16828/consoleFull
---
If
1 - 100 of 192 matches
Mail list logo
