Re: Log4j 1.2.17 spark CVE

[Sean Owen]

FWIW here is the Databricks statement on it. Not the same as Spark but
includes Spark of course.

https://databricks.com/blog/2021/12/13/log4j2-vulnerability-cve-2021-44228-research-and-assessment.html

Yes the question is almost surely more whether user apps are affected, not
Spark itself.

On Tue, Dec 14, 2021, 7:55 AM Steve Loughran 
wrote:

> log4j 1.2.17 is not vulnerable. There is an existing CVE there from a log
> aggregation servlet; Cloudera products ship a patched release with that
> servlet stripped...asf projects are not allowed to do that.
>
> But: some recent Cloudera Products do include log4j 2.x, so colleagues of
> mine are busy patching and retesting everything. If anyone replaces the
> vulnerable jars themselves, remember to look in spark.tar.gz on hdfs to
> make sure it is safe.
>
>
> hadoop stayed on log4j 1.2.17 because 2.x
> * would have broken all cluster management tools which configured
> log4j.properties files
> * wouldn't let us use System properties to can I figure logging... That is
> really useful when you want to run a job with debug logging
> * didn't support the no capture we use in mockito and functional tests
>
> But: the SLF4J it's used throughout; spark doesn't need to be held back by
> that choice and can use any backend you want
>
> I don't know what we will do now; akira has just suggested logback
> https://issues.apache.org/jira/browse/HADOOP-12956
>
> had I not just broken a collar bone and so unable to code, I would have
> added a new command to audit the the hadoop class path to verify it wasn't
> vulnerable. Someone could do the same for spark -where you would want an
> RDD where the probe would also take place in worker tasks to validate the
> the cluster safety more broadly, including the tarball.
>
> meanwhile, if your product is not exposed -probably worth mentioning on
> the users mailing list so as to help people focus their attention. It's
> probably best to work with everyone who produces spark based Products so
> that you can have a single summary.
>
> On Tue, 14 Dec 2021 at 01:31, Qian Sun  wrote:
>
>> My understanding is that we don’t need to do anything. Log4j2-core not
>> used in spark.
>>
>> > 2021年12月13日 下午12:45，Pralabh Kumar  写道：
>> >
>> > Hi developers,  users
>> >
>> > Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
>> recent CVE detected ?
>> >
>> >
>> > Regards
>> > Pralabh kumar
>>
>>
>> -
>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>
>>

Re: Log4j 1.2.17 spark CVE

[Steve Loughran]

log4j 1.2.17 is not vulnerable. There is an existing CVE there from a log
aggregation servlet; Cloudera products ship a patched release with that
servlet stripped...asf projects are not allowed to do that.

But: some recent Cloudera Products do include log4j 2.x, so colleagues of
mine are busy patching and retesting everything. If anyone replaces the
vulnerable jars themselves, remember to look in spark.tar.gz on hdfs to
make sure it is safe.


hadoop stayed on log4j 1.2.17 because 2.x
* would have broken all cluster management tools which configured
log4j.properties files
* wouldn't let us use System properties to can I figure logging... That is
really useful when you want to run a job with debug logging
* didn't support the no capture we use in mockito and functional tests

But: the SLF4J it's used throughout; spark doesn't need to be held back by
that choice and can use any backend you want

I don't know what we will do now; akira has just suggested logback
https://issues.apache.org/jira/browse/HADOOP-12956

had I not just broken a collar bone and so unable to code, I would have
added a new command to audit the the hadoop class path to verify it wasn't
vulnerable. Someone could do the same for spark -where you would want an
RDD where the probe would also take place in worker tasks to validate the
the cluster safety more broadly, including the tarball.

meanwhile, if your product is not exposed -probably worth mentioning on the
users mailing list so as to help people focus their attention. It's
probably best to work with everyone who produces spark based Products so
that you can have a single summary.

On Tue, 14 Dec 2021 at 01:31, Qian Sun  wrote:

> My understanding is that we don’t need to do anything. Log4j2-core not
> used in spark.
>
> > 2021年12月13日 下午12:45，Pralabh Kumar  写道：
> >
> > Hi developers,  users
> >
> > Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
> recent CVE detected ?
> >
> >
> > Regards
> > Pralabh kumar
>
>
> -
> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>
>

Re: Log4j 1.2.17 spark CVE

[Qian Sun]

My understanding is that we don’t need to do anything. Log4j2-core not used in 
spark.

> 2021年12月13日 下午12:45，Pralabh Kumar  写道：
> 
> Hi developers,  users 
> 
> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on 
> recent CVE detected ?
> 
> 
> Regards
> Pralabh kumar


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Log4j 1.2.17 spark CVE

[Sean Owen]

You would want to shade this dependency in your app, in which case you
would be using log4j 2. If you don't shade and just include it, you will
also be using log4j 2 as some of the API classes are different. If they
overlap with log4j 1, you will probably hit errors anyway.

On Mon, Dec 13, 2021 at 6:33 PM James Yu  wrote:

> Question: Spark use log4j 1.2.17, if my application jar contains log4j 2.x
> and gets submitted to the Spark cluster.  Which version of log4j gets
> actually used during the Spark session?
> --
> *From:* Sean Owen 
> *Sent:* Monday, December 13, 2021 8:25 AM
> *To:* Jörn Franke 
> *Cc:* Pralabh Kumar ; dev ;
> user.spark 
> *Subject:* Re: Log4j 1.2.17 spark CVE
>
> This has come up several times over years - search JIRA. The very short
> summary is: Spark does not use log4j 1.x, but its dependencies do, and
> that's the issue.
> Anyone that can successfully complete the surgery at this point is welcome
> to, but I failed ~2 years ago.
>
> On Mon, Dec 13, 2021 at 10:02 AM Jörn Franke  wrote:
>
> Is it in any case appropriate to use log4j 1.x which is not maintained
> anymore and has other security vulnerabilities which won’t be fixed anymore
> ?
>
> Am 13.12.2021 um 06:06 schrieb Sean Owen :
>
> 
> Check the CVE - the log4j vulnerability appears to affect log4j 2, not
> 1.x. There was mention that it could affect 1.x when used with JNDI or SMS
> handlers, but Spark does neither. (unless anyone can think of something I'm
> missing, but never heard or seen that come up at all in 7 years in Spark)
>
> The big issue would be applications that themselves configure log4j 2.x,
> but that's not a Spark issue per se.
>
> On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar 
> wrote:
>
> Hi developers,  users
>
> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
> recent CVE detected ?
>
>
> Regards
> Pralabh kumar
>
>

Re: Log4j 1.2.17 spark CVE

[James Yu]

Question: Spark use log4j 1.2.17, if my application jar contains log4j 2.x and 
gets submitted to the Spark cluster.  Which version of log4j gets actually used 
during the Spark session?

From: Sean Owen 
Sent: Monday, December 13, 2021 8:25 AM
To: Jörn Franke 
Cc: Pralabh Kumar ; dev ; 
user.spark 
Subject: Re: Log4j 1.2.17 spark CVE

This has come up several times over years - search JIRA. The very short summary 
is: Spark does not use log4j 1.x, but its dependencies do, and that's the issue.
Anyone that can successfully complete the surgery at this point is welcome to, 
but I failed ~2 years ago.

On Mon, Dec 13, 2021 at 10:02 AM Jörn Franke 
mailto:jornfra...@gmail.com>> wrote:
Is it in any case appropriate to use log4j 1.x which is not maintained anymore 
and has other security vulnerabilities which won’t be fixed anymore ?

Am 13.12.2021 um 06:06 schrieb Sean Owen 
mailto:sro...@gmail.com>>:


Check the CVE - the log4j vulnerability appears to affect log4j 2, not 1.x. 
There was mention that it could affect 1.x when used with JNDI or SMS handlers, 
but Spark does neither. (unless anyone can think of something I'm missing, but 
never heard or seen that come up at all in 7 years in Spark)

The big issue would be applications that themselves configure log4j 2.x, but 
that's not a Spark issue per se.

On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar 
mailto:pralabhku...@gmail.com>> wrote:
Hi developers,  users

Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on recent 
CVE detected ?


Regards
Pralabh kumar

Re: Log4j 1.2.17 spark CVE

[Sean Owen]

This has come up several times over years - search JIRA. The very short
summary is: Spark does not use log4j 1.x, but its dependencies do, and
that's the issue.
Anyone that can successfully complete the surgery at this point is welcome
to, but I failed ~2 years ago.

On Mon, Dec 13, 2021 at 10:02 AM Jörn Franke  wrote:

> Is it in any case appropriate to use log4j 1.x which is not maintained
> anymore and has other security vulnerabilities which won’t be fixed anymore
> ?
>
> Am 13.12.2021 um 06:06 schrieb Sean Owen :
>
> 
> Check the CVE - the log4j vulnerability appears to affect log4j 2, not
> 1.x. There was mention that it could affect 1.x when used with JNDI or SMS
> handlers, but Spark does neither. (unless anyone can think of something I'm
> missing, but never heard or seen that come up at all in 7 years in Spark)
>
> The big issue would be applications that themselves configure log4j 2.x,
> but that's not a Spark issue per se.
>
> On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar 
> wrote:
>
>> Hi developers,  users
>>
>> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
>> recent CVE detected ?
>>
>>
>> Regards
>> Pralabh kumar
>>
>

Re: Log4j 1.2.17 spark CVE

[Martin Wunderlich]

There is a discussion on Github on this topic and the recommendation is 
to upgrade from 1.x to 2.15.0, due to the vulnerability of 1.x: 
https://github.com/apache/logging-log4j2/pull/608


This discussion is also referenced by the German Federal Office for 
Information Security: https://www.bsi.bund.de/EN/Home/home_node.html


Cheers,

Martin

Am 13.12.21 um 17:02 schrieb Jörn Franke:
Is it in any case appropriate to use log4j 1.x which is not maintained 
anymore and has other security vulnerabilities which won’t be fixed 
anymore ?



Am 13.12.2021 um 06:06 schrieb Sean Owen :


Check the CVE - the log4j vulnerability appears to affect log4j 2, 
not 1.x. There was mention that it could affect 1.x when used with 
JNDI or SMS handlers, but Spark does neither. (unless anyone can 
think of something I'm missing, but never heard or seen that come up 
at all in 7 years in Spark)


The big issue would be applications that themselves configure log4j 
2.x, but that's not a Spark issue per se.


On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar 
 wrote:


Hi developers,  users

Spark is built using log4j 1.2.17 . Is there a plan to upgrade
based on recent CVE detected ?


Regards
Pralabh kumar

Re: Log4j 1.2.17 spark CVE

[Jörn Franke]

Is it in any case appropriate to use log4j 1.x which is not maintained anymore 
and has other security vulnerabilities which won’t be fixed anymore ?

> Am 13.12.2021 um 06:06 schrieb Sean Owen :
> 
> 
> Check the CVE - the log4j vulnerability appears to affect log4j 2, not 1.x. 
> There was mention that it could affect 1.x when used with JNDI or SMS 
> handlers, but Spark does neither. (unless anyone can think of something I'm 
> missing, but never heard or seen that come up at all in 7 years in Spark)
> 
> The big issue would be applications that themselves configure log4j 2.x, but 
> that's not a Spark issue per se.
> 
>> On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar  
>> wrote:
>> Hi developers,  users 
>> 
>> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on 
>> recent CVE detected ?
>> 
>> 
>> Regards
>> Pralabh kumar

Re: Log4j 1.2.17 spark CVE

[Holden Karau]

My understanding is it only applies to log4j 2+ so we don’t need to do
anything.

On Sun, Dec 12, 2021 at 8:46 PM Pralabh Kumar 
wrote:

> Hi developers,  users
>
> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
> recent CVE detected ?
>
>
> Regards
> Pralabh kumar
>
-- 
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Re: Log4j 1.2.17 spark CVE

[Sean Owen]

Check the CVE - the log4j vulnerability appears to affect log4j 2, not 1.x.
There was mention that it could affect 1.x when used with JNDI or SMS
handlers, but Spark does neither. (unless anyone can think of something I'm
missing, but never heard or seen that come up at all in 7 years in Spark)

The big issue would be applications that themselves configure log4j 2.x,
but that's not a Spark issue per se.

On Sun, Dec 12, 2021 at 10:46 PM Pralabh Kumar 
wrote:

> Hi developers,  users
>
> Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
> recent CVE detected ?
>
>
> Regards
> Pralabh kumar
>

Log4j 1.2.17 spark CVE

[Pralabh Kumar]

Hi developers,  users

Spark is built using log4j 1.2.17 . Is there a plan to upgrade based on
recent CVE detected ?


Regards
Pralabh kumar