lordk911 opened a new issue #1175:
URL: https://github.com/apache/incubator-linkis/issues/1175
**I'm using Linkis 1.0.2 and Spark3.1.2, When I try to use pyspark , I got
ERROR:**
```
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: Traceback (most recent
call last):
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: File
"/data/dss-dir/appcom/tmp/bigtop/workDir/422513a8-1eda-4449-8c79-eed7b6b82ae6/tmp/2078255478639789418.py
", line 164, in <module>
21/12/09 13:11:21 INFO executor.SparkPythonExecutor:
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: sc =
SparkContext(jsc=jsc, gateway=gateway, conf=conf)
21/12/09 13:11:21 INFO executor.SparkPythonExecutor:
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: File
"/data/soft/spark3/python/lib/pyspark.zip/pyspark/context.py", line 140, in
__init__
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: ValueError
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: :
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: You are trying to pass
an insecure Py4j gateway to Spark. This is not allowed as it is a security risk.
21/12/09 13:11:21 INFO executor.SparkPythonExecutor:
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: Pyspark process has
stopped with exit code 1
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: python executor ready
to close
21/12/09 13:11:21 INFO executor.SparkPythonExecutor: python executor
Finished to close
21/12/09 13:11:51 ERROR executor.SparkPythonExecutor: initialize python
executor failed, please ask administrator for help!
java.util.concurrent.TimeoutException
at
com.webank.wedatasphere.linkis.common.utils.Utils$.aux$1(Utils.scala:193)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.waitUntil(Utils.scala:198)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.waitUntil(Utils.scala:201)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkPythonExecutor.initGateway(SparkPythonExecutor.scala:193)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkPythonExecutor.$anonfun$lazyInitGageWay$1(SparkPythonExecutor.scala:209)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:39)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryThrow(Utils.scala:57)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkPythonExecutor.lazyInitGageWay(SparkPythonExecutor.scala:209)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkPythonExecutor.runCode(SparkPythonExecutor.scala:202)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkEngineConnExecutor.$anonfun$executeLine$7(SparkEngineConnExecutor.scala:87)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkEngineConnExecutor.$anonfun$executeLine$1(SparkEngineConnExecutor.scala:87)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60)
at
com.webank.wedatasphere.linkis.engineplugin.spark.executor.SparkEngineConnExecutor.executeLine(SparkEngineConnExecutor.scala:95)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.$anonfun$toExecuteTask$14(ComputationExecutor.scala:179)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:39)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.$anonfun$toExecuteTask$13(ComputationExecutor.scala:181)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.$anonfun$toExecuteTask$13$adapted(ComputationExecutor.scala:17
4)
at scala.collection.immutable.Range.foreach(Range.scala:158)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.$anonfun$toExecuteTask$1(ComputationExecutor.scala:174)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.toExecuteTask(ComputationExecutor.scala:222)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.$anonfun$execute$2(ComputationExecutor.scala:237)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryFinally(Utils.scala:60)
at
com.webank.wedatasphere.linkis.engineconn.acessible.executor.entity.AccessibleExecutor.ensureIdle(AccessibleExecutor.scala:54)
at
com.webank.wedatasphere.linkis.engineconn.acessible.executor.entity.AccessibleExecutor.ensureIdle(AccessibleExecutor.scala:48)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.ensureOp(ComputationExecutor.scala:133)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.execute.ComputationExecutor.execute(ComputationExecutor.scala:237)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl.com$webank$wedatasphere$linkis$engineconn$computation$exe
cutor$service$TaskExecutionServiceImpl$$executeTask(TaskExecutionServiceImpl.scala:239)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1.$anonfun$run$1(TaskExecutionServiceImpl.scala:172
)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryCatch(Utils.scala:39)
at
com.webank.wedatasphere.linkis.common.utils.Utils$.tryAndWarn(Utils.scala:68)
at
com.webank.wedatasphere.linkis.engineconn.computation.executor.service.TaskExecutionServiceImpl$$anon$1.run(TaskExecutionServiceImpl.scala:170)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21/12/09 13:11:52 ERROR executor.SparkPythonExecutor: execute code failed!
```
**I think** `21/12/09 13:11:21 INFO executor.SparkPythonExecutor: You are
trying to pass an insecure Py4j gateway to Spark. This is not allowed as it is
a security risk.
` **is the root cause.**
When or how this issue could be fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
