Re: Java SPI jar reload in Spark

2017-06-07 Thread Jonnas Li(Contractor)
<mailto:user@spark.apache.org>> 主题: Re: Java SPI jar reload in Spark Hi, a quick search on google. https://github.com/spark-jobserver/spark-jobserver/issues/130 <https://about.me/alonso.isidoro.roman?promo=email_sig_source=email_sig_medium=email_sig_campaign=external_lin

Re: Java SPI jar reload in Spark

2017-06-06 Thread Jonnas Li(Contractor)
links> Alonso Isidoro Roman about.me/alonso.isidoro.roman 2017-06-06 12:14 GMT+02:00 Jonnas Li(Contractor) <zhongshuang...@envisioncn.com<mailto:zhongshuang...@envisioncn.com>>: Thank for your quick response. These jars are used to define some customize business logic, and they

Re: Java SPI jar reload in Spark

2017-06-06 Thread Jonnas Li(Contractor)
there is another way to achieve the same without jar reloading. In fact, it might be dangerous from a functional point of view- functionality in jar changed and all your computation is wrong. On 6. Jun 2017, at 11:35, Jonnas Li(Contractor) <zhongshuang...@envisioncn.com<mailto:zhongshuang...@env

Java SPI jar reload in Spark

2017-06-06 Thread Jonnas Li(Contractor)
I have a Spark Streaming application, which dynamically calling a jar (Java SPI), and the jar is called in a mapWithState() function, it was working fine for a long time. Recently, I got a requirement which required to reload the jar during runtime. But when the reloading is completed, the spark