Re: [DISCUSS] Switching Phoenix to log4j2

[Josh Elser]

Agree on your solution proposed, Istvan.

I think a Phoenix 5.2 is the right time to take that on, too.

On 4/26/22 2:21 PM, Andrew Purtell wrote:

Thanks, I understand better.


What I am proposing is keeping phoenix-client-embedded, but dropping the

legacy (non embedded) phoenix-client jar/artifact from 5.2.

+1, for what it's worth. Embedding a logging back end is a bad idea as we
have learned. Only the facade (SLF4J) should be necessary.


On Tue, Apr 26, 2022 at 11:14 AM Istvan Toth 
wrote:


Andrew, what you describe is the phoenix-client-embedded jar, and it is the
(or at least my) preferred way to consume the phoenix thick client.

However, we still build and publish the legacy phoenix-client (non
embedded) JAR, that DOES include the slf4j + logging backend libraries (as
well as sqlline + jline)

What I am proposing is keeping phoenix-client-embedded, but dropping the
legacy (non embedded) phoenix-client jar/artifact from 5.2.

sqlline.py and friends used to use the non-embedded jar, so that they get
logging and sqlline, but I have since modified all scripts to use the
embedded client, and add the logging backend and sqlline from /lib, so
nothing we ship depends on the legacy phoenix-client JAR any longer.

regards
Istvan

Re: [DISCUSS] Switching Phoenix to log4j2

[Andrew Purtell]

Thanks, I understand better.

> What I am proposing is keeping phoenix-client-embedded, but dropping the
legacy (non embedded) phoenix-client jar/artifact from 5.2.

+1, for what it's worth. Embedding a logging back end is a bad idea as we
have learned. Only the facade (SLF4J) should be necessary.


On Tue, Apr 26, 2022 at 11:14 AM Istvan Toth 
wrote:

> Andrew, what you describe is the phoenix-client-embedded jar, and it is the
> (or at least my) preferred way to consume the phoenix thick client.
>
> However, we still build and publish the legacy phoenix-client (non
> embedded) JAR, that DOES include the slf4j + logging backend libraries (as
> well as sqlline + jline)
>
> What I am proposing is keeping phoenix-client-embedded, but dropping the
> legacy (non embedded) phoenix-client jar/artifact from 5.2.
>
> sqlline.py and friends used to use the non-embedded jar, so that they get
> logging and sqlline, but I have since modified all scripts to use the
> embedded client, and add the logging backend and sqlline from /lib, so
> nothing we ship depends on the legacy phoenix-client JAR any longer.
>
> regards
> Istvan
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: [DISCUSS] Switching Phoenix to log4j2

[Istvan Toth]

Andrew, what you describe is the phoenix-client-embedded jar, and it is the
(or at least my) preferred way to consume the phoenix thick client.

However, we still build and publish the legacy phoenix-client (non
embedded) JAR, that DOES include the slf4j + logging backend libraries (as
well as sqlline + jline)

What I am proposing is keeping phoenix-client-embedded, but dropping the
legacy (non embedded) phoenix-client jar/artifact from 5.2.

sqlline.py and friends used to use the non-embedded jar, so that they get
logging and sqlline, but I have since modified all scripts to use the
embedded client, and add the logging backend and sqlline from /lib, so
nothing we ship depends on the legacy phoenix-client JAR any longer.

regards
Istvan

Re: [DISCUSS] Switching Phoenix to log4j2

[Andrew Purtell]

It shouldn't be necessary to shade the logger, if you are using SLF4J
everywhere as logging facade in Phoenix, as I believe you are. Exclude
log4j and reload4j for shading into the client jar, and supply them in lib/
instead in your convenience artifact. The user can opt to include them on
the classpath or provide their own solution. They are required to have some
SLF4J implementation jar on the classpath already by HBase etc, but it can
be whatever they decide.

On Mon, Apr 25, 2022 at 10:33 PM Istvan Toth  wrote:

> Another issue that came to my mind is the phoenix-client jar.
> We're currently shading log4j1 (actually reload4j) into the JAR.
>
> Our options are (in the order of my preference):
> * drop phoenix-client (and make users migrate to phoenix-client-embedded)
> * switch phoenix-client to log4j2
> * we could also keep shading reload4j into it, but that would defeat the
> purpose of switching backends.
>
> If we change the logging backend in phoenix-client, we're already changing
> the dependent application's logging environment,
> so users may as well  switch to phoenix-client-embedded, and take control
> of their backend.
>
> Istvan
>
> On Mon, Apr 25, 2022 at 6:47 PM Istvan Toth  wrote:
>
> > Thanks, Geoffrey.
> >
> > I actually plan to push the logger changes in a separate JIRA/Commit.
> > Seems like the Hbase 2.5 artifacts don't actually depend on any logging
> > backend (which is nice),
> > so the current log4j1 backend works fine with minimal (test only) pom
> > changes.
> >
> >
> >
> > On Mon, Apr 25, 2022 at 5:50 PM Geoffrey Jacoby 
> > wrote:
> >
> >> It's a tough question because my instinct is to run whatever logging
> >> system
> >> HBase uses, and that's reload4j in 2.4 and log4j2 in HBase 2.5.
> >>
> >> But trying to have it both ways with complicated profile magic seems
> like
> >> it would add a lot of complexity, likely more than it's worth.
> >>
> >> Since log4j2 is the "future-proof" option that HBase is going to support
> >> not just in 2.5 but in any future 2.6 or 3.0 going forward, I'm +1 for
> >> moving to log4j2 as part of adding HBase 2.5 support.
> >>
> >> Geoffrey
> >>
> >> On Mon, Apr 25, 2022 at 4:14 AM Istvan Toth  wrote:
> >>
> >> > Hi!
> >> >
> >> > We've had a discussion
> >> > <
> https://lists.apache.org/list?dev@phoenix.apache.org:2022-1:[DISCUSS]>
> >> on
> >> > the replacement of log4j a few months ago.
> >> > There were not a lot of replies, but based on Andrew's suggestion
> we've
> >> > switched Phoenix over to reload4j as a short-term fix (as that doesn't
> >> > require any changes by users).
> >> >
> >> > However, we want to support the upcoming HBase 2.5 in Phoenix 5.2
> >> > (PHOENIX-6692), and the log4j changes have recently been backported to
> >> > branch-2.5, meaning that HBase 2.5 no longer uses log4j1 at all.
> >> >
> >> > This adds some urgency to choosing a "final" solution in Phoenix.
> >> >
> >> > Being built on top of HBase, I propose that we simply follow HBase's
> >> lead,
> >> > and go with log4j2, as not having to work with two logging backends is
> >> the
> >> > easiest both for us as developers and for our users.
> >> >
> >> > Looking at the HBase changes, Hbase pulls in
> >> > org.apache.logging.log4j:log4j-1.2-api , which means log4j2 can also
> >> handle
> >> > any log4j using components (hadoop, zookeeper), and a single log4j2
> >> config
> >> > file should be enough for the whole stack.
> >> >
> >> > The changes would look like something like this:
> >> >
> >> > - Explicitly exclude the log4j (and sl4j-logj12) libraries from all
> >> > dependencies
> >> > - Explicitily add the same log4j2 dependencies as HBase to
> phoenix-core
> >> > (and remove reload4j)
> >> > - Replace the log4j config file in the assembly and tests with a
> log4j2
> >> > config file
> >> > - Modify the startup scripts.
> >> > - Add the logging libraries to the assemblies (and remove reload4j)
> >> >
> >> > This needs to be done in Omid, the queryserver, and perhaps in the
> >> > connector repos as well.
> >> >
> >> > WDYT ?
> >> >
> >> > Istvan
> >> >
> >>
> >
> >
> > --
> > *István Tóth* | Staff Software Engineer
> > st...@cloudera.com 
> > [image: Cloudera] 
> > [image: Cloudera on Twitter]  [image:
> > Cloudera on Facebook]  [image:
> > Cloudera on LinkedIn] 
> > 
> > --
> >
>


-- 
Best regards,
Andrew

Unrest, ignorance distilled, nihilistic imbeciles -
It's what we’ve earned
Welcome, apocalypse, what’s taken you so long?
Bring us the fitting end that we’ve been counting on
   - A23, Welcome, Apocalypse

Re: [DISCUSS] Switching Phoenix to log4j2

[Istvan Toth]

Another issue that came to my mind is the phoenix-client jar.
We're currently shading log4j1 (actually reload4j) into the JAR.

Our options are (in the order of my preference):
* drop phoenix-client (and make users migrate to phoenix-client-embedded)
* switch phoenix-client to log4j2
* we could also keep shading reload4j into it, but that would defeat the
purpose of switching backends.

If we change the logging backend in phoenix-client, we're already changing
the dependent application's logging environment,
so users may as well  switch to phoenix-client-embedded, and take control
of their backend.

Istvan

On Mon, Apr 25, 2022 at 6:47 PM Istvan Toth  wrote:

> Thanks, Geoffrey.
>
> I actually plan to push the logger changes in a separate JIRA/Commit.
> Seems like the Hbase 2.5 artifacts don't actually depend on any logging
> backend (which is nice),
> so the current log4j1 backend works fine with minimal (test only) pom
> changes.
>
>
>
> On Mon, Apr 25, 2022 at 5:50 PM Geoffrey Jacoby 
> wrote:
>
>> It's a tough question because my instinct is to run whatever logging
>> system
>> HBase uses, and that's reload4j in 2.4 and log4j2 in HBase 2.5.
>>
>> But trying to have it both ways with complicated profile magic seems like
>> it would add a lot of complexity, likely more than it's worth.
>>
>> Since log4j2 is the "future-proof" option that HBase is going to support
>> not just in 2.5 but in any future 2.6 or 3.0 going forward, I'm +1 for
>> moving to log4j2 as part of adding HBase 2.5 support.
>>
>> Geoffrey
>>
>> On Mon, Apr 25, 2022 at 4:14 AM Istvan Toth  wrote:
>>
>> > Hi!
>> >
>> > We've had a discussion
>> > 
>> on
>> > the replacement of log4j a few months ago.
>> > There were not a lot of replies, but based on Andrew's suggestion we've
>> > switched Phoenix over to reload4j as a short-term fix (as that doesn't
>> > require any changes by users).
>> >
>> > However, we want to support the upcoming HBase 2.5 in Phoenix 5.2
>> > (PHOENIX-6692), and the log4j changes have recently been backported to
>> > branch-2.5, meaning that HBase 2.5 no longer uses log4j1 at all.
>> >
>> > This adds some urgency to choosing a "final" solution in Phoenix.
>> >
>> > Being built on top of HBase, I propose that we simply follow HBase's
>> lead,
>> > and go with log4j2, as not having to work with two logging backends is
>> the
>> > easiest both for us as developers and for our users.
>> >
>> > Looking at the HBase changes, Hbase pulls in
>> > org.apache.logging.log4j:log4j-1.2-api , which means log4j2 can also
>> handle
>> > any log4j using components (hadoop, zookeeper), and a single log4j2
>> config
>> > file should be enough for the whole stack.
>> >
>> > The changes would look like something like this:
>> >
>> > - Explicitly exclude the log4j (and sl4j-logj12) libraries from all
>> > dependencies
>> > - Explicitily add the same log4j2 dependencies as HBase to phoenix-core
>> > (and remove reload4j)
>> > - Replace the log4j config file in the assembly and tests with a log4j2
>> > config file
>> > - Modify the startup scripts.
>> > - Add the logging libraries to the assemblies (and remove reload4j)
>> >
>> > This needs to be done in Omid, the queryserver, and perhaps in the
>> > connector repos as well.
>> >
>> > WDYT ?
>> >
>> > Istvan
>> >
>>
>
>
> --
> *István Tóth* | Staff Software Engineer
> st...@cloudera.com 
> [image: Cloudera] 
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image:
> Cloudera on LinkedIn] 
> 
> --
>

Re: [DISCUSS] Switching Phoenix to log4j2

[Istvan Toth]

Thanks, Geoffrey.

I actually plan to push the logger changes in a separate JIRA/Commit.
Seems like the Hbase 2.5 artifacts don't actually depend on any logging
backend (which is nice),
so the current log4j1 backend works fine with minimal (test only) pom
changes.



On Mon, Apr 25, 2022 at 5:50 PM Geoffrey Jacoby  wrote:

> It's a tough question because my instinct is to run whatever logging system
> HBase uses, and that's reload4j in 2.4 and log4j2 in HBase 2.5.
>
> But trying to have it both ways with complicated profile magic seems like
> it would add a lot of complexity, likely more than it's worth.
>
> Since log4j2 is the "future-proof" option that HBase is going to support
> not just in 2.5 but in any future 2.6 or 3.0 going forward, I'm +1 for
> moving to log4j2 as part of adding HBase 2.5 support.
>
> Geoffrey
>
> On Mon, Apr 25, 2022 at 4:14 AM Istvan Toth  wrote:
>
> > Hi!
> >
> > We've had a discussion
> > 
> on
> > the replacement of log4j a few months ago.
> > There were not a lot of replies, but based on Andrew's suggestion we've
> > switched Phoenix over to reload4j as a short-term fix (as that doesn't
> > require any changes by users).
> >
> > However, we want to support the upcoming HBase 2.5 in Phoenix 5.2
> > (PHOENIX-6692), and the log4j changes have recently been backported to
> > branch-2.5, meaning that HBase 2.5 no longer uses log4j1 at all.
> >
> > This adds some urgency to choosing a "final" solution in Phoenix.
> >
> > Being built on top of HBase, I propose that we simply follow HBase's
> lead,
> > and go with log4j2, as not having to work with two logging backends is
> the
> > easiest both for us as developers and for our users.
> >
> > Looking at the HBase changes, Hbase pulls in
> > org.apache.logging.log4j:log4j-1.2-api , which means log4j2 can also
> handle
> > any log4j using components (hadoop, zookeeper), and a single log4j2
> config
> > file should be enough for the whole stack.
> >
> > The changes would look like something like this:
> >
> > - Explicitly exclude the log4j (and sl4j-logj12) libraries from all
> > dependencies
> > - Explicitily add the same log4j2 dependencies as HBase to phoenix-core
> > (and remove reload4j)
> > - Replace the log4j config file in the assembly and tests with a log4j2
> > config file
> > - Modify the startup scripts.
> > - Add the logging libraries to the assemblies (and remove reload4j)
> >
> > This needs to be done in Omid, the queryserver, and perhaps in the
> > connector repos as well.
> >
> > WDYT ?
> >
> > Istvan
> >
>


-- 
*István Tóth* | Staff Software Engineer
st...@cloudera.com 
[image: Cloudera] 
[image: Cloudera on Twitter]  [image:
Cloudera on Facebook]  [image: Cloudera
on LinkedIn] 

--

Re: [DISCUSS] Switching Phoenix to log4j2

[Geoffrey Jacoby]

It's a tough question because my instinct is to run whatever logging system
HBase uses, and that's reload4j in 2.4 and log4j2 in HBase 2.5.

But trying to have it both ways with complicated profile magic seems like
it would add a lot of complexity, likely more than it's worth.

Since log4j2 is the "future-proof" option that HBase is going to support
not just in 2.5 but in any future 2.6 or 3.0 going forward, I'm +1 for
moving to log4j2 as part of adding HBase 2.5 support.

Geoffrey

On Mon, Apr 25, 2022 at 4:14 AM Istvan Toth  wrote:

> Hi!
>
> We've had a discussion
>  on
> the replacement of log4j a few months ago.
> There were not a lot of replies, but based on Andrew's suggestion we've
> switched Phoenix over to reload4j as a short-term fix (as that doesn't
> require any changes by users).
>
> However, we want to support the upcoming HBase 2.5 in Phoenix 5.2
> (PHOENIX-6692), and the log4j changes have recently been backported to
> branch-2.5, meaning that HBase 2.5 no longer uses log4j1 at all.
>
> This adds some urgency to choosing a "final" solution in Phoenix.
>
> Being built on top of HBase, I propose that we simply follow HBase's lead,
> and go with log4j2, as not having to work with two logging backends is the
> easiest both for us as developers and for our users.
>
> Looking at the HBase changes, Hbase pulls in
> org.apache.logging.log4j:log4j-1.2-api , which means log4j2 can also handle
> any log4j using components (hadoop, zookeeper), and a single log4j2 config
> file should be enough for the whole stack.
>
> The changes would look like something like this:
>
> - Explicitly exclude the log4j (and sl4j-logj12) libraries from all
> dependencies
> - Explicitily add the same log4j2 dependencies as HBase to phoenix-core
> (and remove reload4j)
> - Replace the log4j config file in the assembly and tests with a log4j2
> config file
> - Modify the startup scripts.
> - Add the logging libraries to the assemblies (and remove reload4j)
>
> This needs to be done in Omid, the queryserver, and perhaps in the
> connector repos as well.
>
> WDYT ?
>
> Istvan
>

[DISCUSS] Switching Phoenix to log4j2

[Istvan Toth]

Hi!

We've had a discussion
 on
the replacement of log4j a few months ago.
There were not a lot of replies, but based on Andrew's suggestion we've
switched Phoenix over to reload4j as a short-term fix (as that doesn't
require any changes by users).

However, we want to support the upcoming HBase 2.5 in Phoenix 5.2
(PHOENIX-6692), and the log4j changes have recently been backported to
branch-2.5, meaning that HBase 2.5 no longer uses log4j1 at all.

This adds some urgency to choosing a "final" solution in Phoenix.

Being built on top of HBase, I propose that we simply follow HBase's lead,
and go with log4j2, as not having to work with two logging backends is the
easiest both for us as developers and for our users.

Looking at the HBase changes, Hbase pulls in
org.apache.logging.log4j:log4j-1.2-api , which means log4j2 can also handle
any log4j using components (hadoop, zookeeper), and a single log4j2 config
file should be enough for the whole stack.

The changes would look like something like this:

- Explicitly exclude the log4j (and sl4j-logj12) libraries from all
dependencies
- Explicitily add the same log4j2 dependencies as HBase to phoenix-core
(and remove reload4j)
- Replace the log4j config file in the assembly and tests with a log4j2
config file
- Modify the startup scripts.
- Add the logging libraries to the assemblies (and remove reload4j)

This needs to be done in Omid, the queryserver, and perhaps in the
connector repos as well.

WDYT ?

Istvan