[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177084#comment-17177084
]
Kayal commented on SPARK-32053:
---
~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-packages\py4j\protocol.py
in get_return_value(answer, gateway_client, target_id, name)
327 "An error occurred while calling \{0}{1}\{2}.\n".
--> 328 format(target_id, ".", name), value)
329 else:
Py4JJavaError: An error occurred while calling o289.save.
: org.apache.spark.SparkException: Job aborted.
at
org.apache.spark.internal.io.SparkHadoopWriter$.write(SparkHadoopWriter.scala:100)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopDataset$1(PairRDDFunctions.scala:1090)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2417.1D19A7B0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopDataset(PairRDDFunctions.scala:1088)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$4(PairRDDFunctions.scala:1061)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2415.0FE34B70.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:1026)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$3(PairRDDFunctions.scala:1008)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2414.1CBB0D40.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:1007)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$2(PairRDDFunctions.scala:964)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2413.1D196EA0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:962)
at org.apache.spark.rdd.RDD.$anonfun$saveAsTextFile$2(RDD.scala:1552)
at org.apache.spark.rdd.RDD$$Lambda$2411.18FEB4E0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1552)
at org.apache.spark.rdd.RDD.$anonfun$saveAsTextFile$1(RDD.scala:1538)
at org.apache.spark.rdd.RDD$$Lambda$2410.1CA30180.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1538)
at
org.apache.spark.ml.util.DefaultParamsWriter$.saveMetadata(ReadWrite.scala:413)
at
org.apache.spark.ml.Pipeline$SharedReadWrite$.$anonfun$saveImpl$1(Pipeline.scala:250)
at
org.apache.spark.ml.Pipeline$SharedReadWrite$.$anonfun$saveImpl$1$adapted(Pipeline.scala:247)
at
org.apache.spark.ml.Pipeline$SharedReadWrite$$$Lambda$2397.190AB010.apply(Unknown
Source)
at
org.apache.spark.ml.util.Instrumentation$.$anonfun$instrumented$1(Instrumentation.scala:191)
at
org.apache.spark.ml.util.Instrumentation$$$Lambda$1390.18680E40.apply(Unknown
Source)
at scala.util.Try$.apply(Try.scala:213)
at
org.apache.spark.ml.util.Instrumentation$.instrumented(Instrumentation.scala:191)
at org.apache.spark.ml.Pipeline$SharedReadWrite$.saveImpl(Pipeline.scala:247)
at org.apache.spark.ml.Pipeline$PipelineWriter.saveImpl(Pipeline.scala:206)
at org.apache.spark.ml.util.MLWriter.save(ReadWrite.scala:168)
at
[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177082#comment-17177082
]
Kayal commented on SPARK-32053:
---
!image-2020-08-13-20-29-40-555.png!
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png, image-2020-08-13-20-25-57-585.png,
> image-2020-08-13-20-28-28-779.png, image-2020-08-13-20-29-40-555.png,
> screenshot-1.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: image-2020-08-13-20-29-40-555.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png, image-2020-08-13-20-25-57-585.png,
> image-2020-08-13-20-28-28-779.png, image-2020-08-13-20-29-40-555.png,
> screenshot-1.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: image-2020-08-13-20-28-28-779.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png, image-2020-08-13-20-25-57-585.png,
> image-2020-08-13-20-28-28-779.png, screenshot-1.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: screenshot-1.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png, image-2020-08-13-20-25-57-585.png,
> screenshot-1.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177079#comment-17177079
]
Kayal commented on SPARK-32053:
---
!image-2020-08-13-20-25-57-585.png!
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: image-2020-08-13-20-25-57-585.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png, image-2020-08-13-20-25-57-585.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: image-2020-08-13-20-24-57-309.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png,
> image-2020-08-13-20-24-57-309.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177077#comment-17177077
]
Kayal commented on SPARK-32053:
---
!image-2020-08-13-20-24-57-309.png!
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177073#comment-17177073
]
Kayal commented on SPARK-32053:
---
The code to reproduce the issue on windows jupyter notebook:
import pyspark
#from pyspark.sql import SQLContext
from pyspark import SparkContext
sc = SparkContext("local", "First App")
from pyspark.sql import SparkSession
sess = SparkSession(sc)
training = sess.createDataFrame([
("0L", "a b c d e WML", 1.0),
("1L", "b d", 0.0),
("2L", "WML f g h", 1.0),
("3L", "hadoop mapreduce", 0.0)], ["id", "text", "label"])
evaluation = sess.createDataFrame([
("4L", "a b c WML", 1.0),
("5L", "l m n o p", 0.0),
("6L", "WML g h i k", 1.0),
("7L", "apache hadoop zuzu", 0.0)], ["id", "text", "label"])
testing = sess.createDataFrame([
("4L", "a b c z WML"),
("5L", "l m n"),
("6L", "WML g h i j k"),
("7L", "apache hadoop")], ["id", "text"])
import traceback
from pyspark.ml.pipeline import Pipeline
from pyspark.ml.classification import LogisticRegression
from pyspark.ml.feature import HashingTF, Tokenizer
from pyspark.sql import SQLContext as sql_context
tokenizer = Tokenizer(inputCol="text", outputCol="words")
hashingTF = HashingTF(inputCol=tokenizer.getOutputCol(), outputCol="features")
lr = LogisticRegression(maxIter=10, regParam=0.01)
stages=[tokenizer, hashingTF, lr]
pipeline = Pipeline(stages=stages)
model = pipeline.fit(training)
test_result = model.transform(testing)
pipeline.write().overwrite().save("tempfile")
The write operation is failing with the error that I mentioned above. This is
blocking our product delivery. could consider this with high priority blocker
issue. Is there a work around for this ? sparkml is supported on windows
pyspark ?
I also noticed the same error with
pipline.save() method.
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Major
> Attachments: image-2020-06-22-18-19-32-236.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Priority: Blocker (was: Major)
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Blocker
> Attachments: image-2020-06-22-18-19-32-236.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal reopened SPARK-32053:
---
Hi,
I have verified the issue in spark latest version 3.0.0 , the issue seems to be
still there on windows.
The problem is on windows when we try to
pipline.write().overwrite().save(temp_dir) is failing with
~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-packages\pyspark\ml\util.py
in save(self, path)
173 if not isinstance(path, basestring):
174 raise TypeError("path should be a basestring, got type %s" % type(path))
--> 175 self._jwrite.save(path)
176
177 def overwrite(self):
~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-packages\py4j\java_gateway.py
in __call__(self, *args)
1303 answer = self.gateway_client.send_command(command)
1304 return_value = get_return_value(
-> 1305 answer, self.gateway_client, self.target_id, self.name)
1306
1307 for temp_arg in temp_args:
~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-packages\pyspark\sql\utils.py
in deco(*a, **kw)
129 def deco(*a, **kw):
130 try:
--> 131 return f(*a, **kw)
132 except py4j.protocol.Py4JJavaError as e:
133 converted = convert_exception(e.java_exception)
~\AppData\Local\IBMWS\miniconda3\envs\desktop\lib\site-packages\py4j\protocol.py
in get_return_value(answer, gateway_client, target_id, name)
326 raise Py4JJavaError(
327 "An error occurred while calling \{0}{1}\{2}.\n".
--> 328 format(target_id, ".", name), value)
329 else:
330 raise Py4JError(
Py4JJavaError: An error occurred while calling o662.save.
: org.apache.spark.SparkException: Job aborted.
at
org.apache.spark.internal.io.SparkHadoopWriter$.write(SparkHadoopWriter.scala:100)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopDataset$1(PairRDDFunctions.scala:1090)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2417.1D19A7B0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopDataset(PairRDDFunctions.scala:1088)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$4(PairRDDFunctions.scala:1061)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2415.0FE34B70.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:1026)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$3(PairRDDFunctions.scala:1008)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2414.1CBB0D40.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:1007)
at
org.apache.spark.rdd.PairRDDFunctions.$anonfun$saveAsHadoopFile$2(PairRDDFunctions.scala:964)
at
org.apache.spark.rdd.PairRDDFunctions$$Lambda$2413.1D196EA0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at
org.apache.spark.rdd.PairRDDFunctions.saveAsHadoopFile(PairRDDFunctions.scala:962)
at org.apache.spark.rdd.RDD.$anonfun$saveAsTextFile$2(RDD.scala:1552)
at org.apache.spark.rdd.RDD$$Lambda$2411.18FEB4E0.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.rdd.RDD.withScope(RDD.scala:388)
at org.apache.spark.rdd.RDD.saveAsTextFile(RDD.scala:1552)
at org.apache.spark.rdd.RDD.$anonfun$saveAsTextFile$1(RDD.scala:1538)
at org.apache.spark.rdd.RDD$$Lambda$2410.1CA30180.apply$mcV$sp(Unknown
Source)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Attachment: image-2020-06-22-18-19-32-236.png
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Major
> Attachments: image-2020-06-22-18-19-32-236.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error : o288.save() failed.
> {color}
>
>
>
> {color:#172b4d}!image-2020-06-22-18-18-36-023.png!{color}
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-32053) pyspark save of serialized model is failing for windows.
[
https://issues.apache.org/jira/browse/SPARK-32053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kayal updated SPARK-32053:
--
Description:
{color:#172b4d}Hi, {color}
{color:#172b4d}We are using spark functionality to save the serialized model to
disk . On windows platform we are seeing save of the serialized model is
failing with the error: o288.save() failed. {color}
!image-2020-06-22-18-19-32-236.png!
was:
{color:#172b4d}Hi, {color}
{color:#172b4d}We are using spark functionality to save the serialized model to
disk . On windows platform we are seeing save of the serialized model is
failing with the error : o288.save() failed.
{color}
{color:#172b4d}!image-2020-06-22-18-18-36-023.png!{color}
> pyspark save of serialized model is failing for windows.
>
>
> Key: SPARK-32053
> URL: https://issues.apache.org/jira/browse/SPARK-32053
> Project: Spark
> Issue Type: Bug
> Components: PySpark
>Affects Versions: 2.3.0
>Reporter: Kayal
>Priority: Major
> Attachments: image-2020-06-22-18-19-32-236.png
>
>
> {color:#172b4d}Hi, {color}
> {color:#172b4d}We are using spark functionality to save the serialized model
> to disk . On windows platform we are seeing save of the serialized model is
> failing with the error: o288.save() failed. {color}
>
>
>
> !image-2020-06-22-18-19-32-236.png!
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-32053) pyspark save of serialized model is failing for windows.
Kayal created SPARK-32053:
-
Summary: pyspark save of serialized model is failing for windows.
Key: SPARK-32053
URL: https://issues.apache.org/jira/browse/SPARK-32053
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 2.3.0
Reporter: Kayal
{color:#172b4d}Hi, {color}
{color:#172b4d}We are using spark functionality to save the serialized model to
disk . On windows platform we are seeing save of the serialized model is
failing with the error : o288.save() failed.
{color}
{color:#172b4d}!image-2020-06-22-18-18-36-023.png!{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
