from:"Andrey Zimovnov \(JIRA\)"

[jira] [Commented] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

2015-08-02 Thread Andrey Zimovnov (JIRA)


[ 
https://issues.apache.org/jira/browse/SPARK-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651067#comment-14651067
 ] 

Andrey Zimovnov commented on SPARK-9539:


Hi, Owen! I'm not sure what Permanent in java heap means, but it grows with 
time. I really have such a use case, when I need to recreate spark context a 
lot. The only workaround for now is to try to increase MaxPermSize, I guess.

 Repeated sc.close() in PySpark causes JVM memory leak
 -

 Key: SPARK-9539
 URL: https://issues.apache.org/jira/browse/SPARK-9539
 Project: Spark
  Issue Type: Bug
  Components: PySpark
Affects Versions: 1.4.1
Reporter: Andrey Zimovnov
Priority: Minor
 Attachments: Screenshot at авг. 02 19-10-53.png


 Example code in Python:
 {code:python}
 for i in range(20):
   print i
   conf = SparkConf().setAppName(test)
   sc = SparkContext(conf=conf)
   hivec = HiveContext(sc)
   hivec.sql(select id from details_info limit 1).show()
   sc.stop()
   del hivec
   del sc
 {code}
 Jstat output:
 {noformat}
  S0CS1CS0US1U  EC   EUOC OU   PC 
 PUYGC YGCTFGCFGCT GCT
 196608,0 196608,0 97566,2  0,0   1179648,0 542150,0 3145728,0120,0
 154112,0 153613,2  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 679041,7 3145728,0120,0
 164352,0 164183,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 907928,4 3145728,0120,0
 164352,0 164200,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 912132,7 3145728,0120,0
 164352,0 164200,5  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 913741,5 3145728,0120,0
 164352,0 164200,8  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 929458,6 3145728,0120,0
 164352,0 164206,0  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 1003138,1 3145728,0120,0
 168960,0 168646,0  40,434   0  0,0000,434
 131584,0 196608,0  0,0   109725,6 1179648,0   0,03145728,0128,0
 175104,0 174802,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 152654,9 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 158586,1 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 160659,8 3145728,0128,0
 175104,0 174805,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 181935,2 3145728,0128,0
 175104,0 174819,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 283389,1 3145728,0128,0
 185856,0 185371,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 342596,4 3145728,0128,0
 185856,0 185379,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 547634,7 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 555930,9 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 557888,6 3145728,0128,0
 185856,0 185386,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 573907,5 3145728,0128,0
 185856,0 185397,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 637955,0 3145728,0128,0
 189952,0 189533,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 895866,1 3145728,0128,0
 196096,0 195968,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 948046,5 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 952427,2 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 957977,5 3145728,0128,0
 196096,0 195973,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 977811,1 3145728,0128,0
 196096,0 195977,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 1118722,0 3145728,0128,0
 206848,0 206539,0  50,591   0  0,0000,591
 131584,0 144384,0 118692,5  0,0   1284096,0 183470,8 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0 118692,5  0,0   1284096,0 189718,5 3145728,0136,0

[jira] [Updated] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

2015-08-02 Thread Andrey Zimovnov (JIRA)


 [ 
https://issues.apache.org/jira/browse/SPARK-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Zimovnov updated SPARK-9539:
---
Description: 
Example code in Python:
{code:python}
for i in range(20):
print i
conf = SparkConf().setAppName(test)
sc = SparkContext(conf=conf)
hivec = HiveContext(sc)
hivec.sql(select id from details_info limit 1).show()
sc.stop()
del hivec
del sc
{code}

Jstat output:
{noformat}
 S0CS1CS0US1U  EC   EUOC OU   PC PU 
   YGC YGCTFGCFGCT GCT
196608,0 196608,0 97566,2  0,0   1179648,0 542150,0 3145728,0120,0
154112,0 153613,2  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 679041,7 3145728,0120,0
164352,0 164183,3  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 907928,4 3145728,0120,0
164352,0 164200,3  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 912132,7 3145728,0120,0
164352,0 164200,5  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 913741,5 3145728,0120,0
164352,0 164200,8  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 929458,6 3145728,0120,0
164352,0 164206,0  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 1003138,1 3145728,0120,0
168960,0 168646,0  40,434   0  0,0000,434
131584,0 196608,0  0,0   109725,6 1179648,0   0,03145728,0128,0
175104,0 174802,1  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 152654,9 3145728,0128,0
175104,0 174803,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 158586,1 3145728,0128,0
175104,0 174803,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 160659,8 3145728,0128,0
175104,0 174805,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 181935,2 3145728,0128,0
175104,0 174819,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 283389,1 3145728,0128,0
185856,0 185371,0  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 342596,4 3145728,0128,0
185856,0 185379,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 547634,7 3145728,0128,0
185856,0 185385,8  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 555930,9 3145728,0128,0
185856,0 185385,8  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 557888,6 3145728,0128,0
185856,0 185386,0  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 573907,5 3145728,0128,0
185856,0 185397,5  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 637955,0 3145728,0128,0
189952,0 189533,1  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 895866,1 3145728,0128,0
196096,0 195968,5  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 948046,5 3145728,0128,0
196096,0 195969,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 952427,2 3145728,0128,0
196096,0 195969,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 957977,5 3145728,0128,0
196096,0 195973,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 977811,1 3145728,0128,0
196096,0 195977,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 1118722,0 3145728,0128,0
206848,0 206539,0  50,591   0  0,0000,591
131584,0 144384,0 118692,5  0,0   1284096,0 183470,8 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 189718,5 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 192165,0 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 199848,4 3145728,0136,0
206848,0 206546,9  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 219687,6 3145728,0136,0
206848,0 206552,2  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 358272,4 3145728,0136,0
217600,0 217100,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 573543,6 3145728,0136,0
217600,0 217109,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0

[jira] [Updated] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

2015-08-02 Thread Andrey Zimovnov (JIRA)


 [ 
https://issues.apache.org/jira/browse/SPARK-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Zimovnov updated SPARK-9539:
---
Attachment: Screenshot at авг. 02 19-10-53.png

jstat visualization

 Repeated sc.close() in PySpark causes JVM memory leak
 -

 Key: SPARK-9539
 URL: https://issues.apache.org/jira/browse/SPARK-9539
 Project: Spark
  Issue Type: Bug
  Components: PySpark
Affects Versions: 1.4.1
Reporter: Andrey Zimovnov
Priority: Minor
 Attachments: Screenshot at авг. 02 19-10-53.png


 Example code in Python:
 {code:python}
 for i in range(20):
   print i
   conf = SparkConf().setAppName(test)
   sc = SparkContext(conf=conf)
   hivec = HiveContext(sc)
   hivec.sql(select id from details_info limit 1).show()
   sc.stop()
   del hivec
   del sc
 {code}
 Jstat output:
 {noformat}
  S0CS1CS0US1U  EC   EUOC OU   PC 
 PUYGC YGCTFGCFGCT GCT
 196608,0 196608,0 97566,2  0,0   1179648,0 542150,0 3145728,0120,0
 154112,0 153613,2  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 679041,7 3145728,0120,0
 164352,0 164183,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 907928,4 3145728,0120,0
 164352,0 164200,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 912132,7 3145728,0120,0
 164352,0 164200,5  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 913741,5 3145728,0120,0
 164352,0 164200,8  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 929458,6 3145728,0120,0
 164352,0 164206,0  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 1003138,1 3145728,0120,0
 168960,0 168646,0  40,434   0  0,0000,434
 131584,0 196608,0  0,0   109725,6 1179648,0   0,03145728,0128,0
 175104,0 174802,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 152654,9 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 158586,1 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 160659,8 3145728,0128,0
 175104,0 174805,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 181935,2 3145728,0128,0
 175104,0 174819,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 283389,1 3145728,0128,0
 185856,0 185371,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 342596,4 3145728,0128,0
 185856,0 185379,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 547634,7 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 555930,9 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 557888,6 3145728,0128,0
 185856,0 185386,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 573907,5 3145728,0128,0
 185856,0 185397,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 637955,0 3145728,0128,0
 189952,0 189533,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 895866,1 3145728,0128,0
 196096,0 195968,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 948046,5 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 952427,2 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 957977,5 3145728,0128,0
 196096,0 195973,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 977811,1 3145728,0128,0
 196096,0 195977,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 1118722,0 3145728,0128,0
 206848,0 206539,0  50,591   0  0,0000,591
 131584,0 144384,0 118692,5  0,0   1284096,0 183470,8 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0 118692,5  0,0   1284096,0 189718,5 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0 118692,5  0,0   1284096,0 192165,0 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0

[jira] [Commented] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

2015-08-02 Thread Andrey Zimovnov (JIRA)


[ 
https://issues.apache.org/jira/browse/SPARK-9539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651078#comment-14651078
 ] 

Andrey Zimovnov commented on SPARK-9539:


OK, I'll work on this later and reopen if necessary. Thanks!

 Repeated sc.close() in PySpark causes JVM memory leak
 -

 Key: SPARK-9539
 URL: https://issues.apache.org/jira/browse/SPARK-9539
 Project: Spark
  Issue Type: Bug
  Components: PySpark
Affects Versions: 1.4.1
Reporter: Andrey Zimovnov
Priority: Minor
 Attachments: Screenshot at авг. 02 19-10-53.png


 Example code in Python:
 {code:python}
 for i in range(20):
   print i
   conf = SparkConf().setAppName(test)
   sc = SparkContext(conf=conf)
   hivec = HiveContext(sc)
   hivec.sql(select id from details_info limit 1).show()
   sc.stop()
   del hivec
   del sc
 {code}
 Jstat output:
 {noformat}
  S0CS1CS0US1U  EC   EUOC OU   PC 
 PUYGC YGCTFGCFGCT GCT
 196608,0 196608,0 97566,2  0,0   1179648,0 542150,0 3145728,0120,0
 154112,0 153613,2  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 679041,7 3145728,0120,0
 164352,0 164183,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 907928,4 3145728,0120,0
 164352,0 164200,3  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 912132,7 3145728,0120,0
 164352,0 164200,5  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 913741,5 3145728,0120,0
 164352,0 164200,8  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 929458,6 3145728,0120,0
 164352,0 164206,0  40,434   0  0,0000,434
 196608,0 196608,0 97566,2  0,0   1179648,0 1003138,1 3145728,0120,0
 168960,0 168646,0  40,434   0  0,0000,434
 131584,0 196608,0  0,0   109725,6 1179648,0   0,03145728,0128,0
 175104,0 174802,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 152654,9 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 158586,1 3145728,0128,0
 175104,0 174803,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 160659,8 3145728,0128,0
 175104,0 174805,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 181935,2 3145728,0128,0
 175104,0 174819,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 283389,1 3145728,0128,0
 185856,0 185371,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 342596,4 3145728,0128,0
 185856,0 185379,3  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 547634,7 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 555930,9 3145728,0128,0
 185856,0 185385,8  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 557888,6 3145728,0128,0
 185856,0 185386,0  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 573907,5 3145728,0128,0
 185856,0 185397,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 637955,0 3145728,0128,0
 189952,0 189533,1  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 895866,1 3145728,0128,0
 196096,0 195968,5  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 948046,5 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 952427,2 3145728,0128,0
 196096,0 195969,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 957977,5 3145728,0128,0
 196096,0 195973,4  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 977811,1 3145728,0128,0
 196096,0 195977,7  50,591   0  0,0000,591
 131584,0 196608,0  0,0   109725,6 1179648,0 1118722,0 3145728,0128,0
 206848,0 206539,0  50,591   0  0,0000,591
 131584,0 144384,0 118692,5  0,0   1284096,0 183470,8 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0 118692,5  0,0   1284096,0 189718,5 3145728,0136,0
 206848,0 206543,4  60,773   0  0,0000,773
 131584,0 144384,0 118692,5  0,0   1284096,0 192165,0 3145728,0136,0
 206848,0 206543,4  6

[jira] [Created] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

2015-08-02 Thread Andrey Zimovnov (JIRA)

Andrey Zimovnov created SPARK-9539:
--

 Summary: Repeated sc.close() in PySpark causes JVM memory leak
 Key: SPARK-9539
 URL: https://issues.apache.org/jira/browse/SPARK-9539
 Project: Spark
  Issue Type: Bug
  Components: PySpark
Affects Versions: 1.4.1
Reporter: Andrey Zimovnov
Priority: Minor


Example code in Python:
for i in range(20):
print i
conf = SparkConf().setAppName(test)
sc = SparkContext(conf=conf)
hivec = HiveContext(sc)
hivec.sql(select id from details_info limit 1).show()
sc.stop()
del hivec
del sc

Jstat output:
 S0CS1CS0US1U  EC   EUOC OU   PC PU 
   YGC YGCTFGCFGCT GCT
196608,0 196608,0 97566,2  0,0   1179648,0 542150,0 3145728,0120,0
154112,0 153613,2  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 679041,7 3145728,0120,0
164352,0 164183,3  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 907928,4 3145728,0120,0
164352,0 164200,3  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 912132,7 3145728,0120,0
164352,0 164200,5  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 913741,5 3145728,0120,0
164352,0 164200,8  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 929458,6 3145728,0120,0
164352,0 164206,0  40,434   0  0,0000,434
196608,0 196608,0 97566,2  0,0   1179648,0 1003138,1 3145728,0120,0
168960,0 168646,0  40,434   0  0,0000,434
131584,0 196608,0  0,0   109725,6 1179648,0   0,03145728,0128,0
175104,0 174802,1  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 152654,9 3145728,0128,0
175104,0 174803,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 158586,1 3145728,0128,0
175104,0 174803,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 160659,8 3145728,0128,0
175104,0 174805,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 181935,2 3145728,0128,0
175104,0 174819,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 283389,1 3145728,0128,0
185856,0 185371,0  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 342596,4 3145728,0128,0
185856,0 185379,3  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 547634,7 3145728,0128,0
185856,0 185385,8  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 555930,9 3145728,0128,0
185856,0 185385,8  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 557888,6 3145728,0128,0
185856,0 185386,0  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 573907,5 3145728,0128,0
185856,0 185397,5  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 637955,0 3145728,0128,0
189952,0 189533,1  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 895866,1 3145728,0128,0
196096,0 195968,5  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 948046,5 3145728,0128,0
196096,0 195969,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 952427,2 3145728,0128,0
196096,0 195969,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 957977,5 3145728,0128,0
196096,0 195973,4  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 977811,1 3145728,0128,0
196096,0 195977,7  50,591   0  0,0000,591
131584,0 196608,0  0,0   109725,6 1179648,0 1118722,0 3145728,0128,0
206848,0 206539,0  50,591   0  0,0000,591
131584,0 144384,0 118692,5  0,0   1284096,0 183470,8 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 189718,5 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 192165,0 3145728,0136,0
206848,0 206543,4  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 199848,4 3145728,0136,0
206848,0 206546,9  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 219687,6 3145728,0136,0
206848,0 206552,2  60,773   0  0,0000,773
131584,0 144384,0 118692,5  0,0   1284096,0 358272,4 3145728,0136,0
217600,0 217100,4  60,773   0  0,000

[jira] [Updated] (SPARK-5271) PySpark History Web UI issues

2015-01-15 Thread Andrey Zimovnov (JIRA)


 [ 
https://issues.apache.org/jira/browse/SPARK-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Zimovnov updated SPARK-5271:
---
Component/s: Web UI

 PySpark History Web UI issues
 -

 Key: SPARK-5271
 URL: https://issues.apache.org/jira/browse/SPARK-5271
 Project: Spark
  Issue Type: Bug
  Components: Web UI
Affects Versions: 1.2.0
 Environment: PySpark 1.2.0 in yarn-client mode
Reporter: Andrey Zimovnov

 After successful run of PySpark app via spark-submit in yarn-client mode on 
 Hadoop 2.4 cluster the History UI shows the same as in issue SPARK-3898.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Created] (SPARK-5271) PySpark History Web UI issues

2015-01-15 Thread Andrey Zimovnov (JIRA)

Andrey Zimovnov created SPARK-5271:
--

 Summary: PySpark History Web UI issues
 Key: SPARK-5271
 URL: https://issues.apache.org/jira/browse/SPARK-5271
 Project: Spark
  Issue Type: Bug
Affects Versions: 1.2.0
 Environment: PySpark 1.2.0 in yarn-client mode
Reporter: Andrey Zimovnov


After successful run of PySpark app via spark-submit in yarn-client mode on 
Hadoop 2.4 cluster the History UI shows the same as in issue SPARK-3898.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

[jira] [Updated] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

[jira] [Updated] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

[jira] [Commented] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

[jira] [Created] (SPARK-9539) Repeated sc.close() in PySpark causes JVM memory leak

[jira] [Updated] (SPARK-5271) PySpark History Web UI issues

[jira] [Created] (SPARK-5271) PySpark History Web UI issues

7 matches

Site Navigation

Mail list logo

Footer information