Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 11.08.2023 11:39, Julian Reschke wrote:

...

Oak itself is now independent of native Guava, but we still use test
tooling that pulls it in (see OAK-10367). Depending on how we resolve
this, we could then remove the remains from config and POMs.


In the meantime, that was fixed by Manfred (thanks!).

However, we have an azure-sdk dependency that transitively pulls in
Guava (see https://issues.apache.org/jira/browse/OAK-10371).
Fortunately, there's a major update to the SDK which does not seem to
use Guava anymore.
...


So, with the release of 1.56.0 last week, this is the only remaining
issue that ties (parts of) Oak to Guava. I believe this should be fixed
in the next release (even if we take the simple way of just embedding it).

If have set the "fixVersion" for OAK-10371 to 1.58.0; can somebody from
the segment persistence experts please have a look ASAP?

Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 26.07.2023 08:28, Julian Reschke wrote:

On 12.06.2023 15:50, Julian Reschke wrote:

...
We have made good progress, but as usual, things take longer than
expected.

We have a pending PR for fixing oak-commons
(https://issues.apache.org/jira/browse/OAK-10247), but I could not apply
it yet (due to the current status of a certain downstream project).

The current plan is to cut 1.54.0 once this has been done (early July),
and then do the remaining changes in 1.56.0.
...


Champagne (sort of).

Oak itself is now independent of native Guava, but we still use test
tooling that pulls it in (see OAK-10367). Depending on how we resolve
this, we could then remove the remains from config and POMs.


In the meantime, that was fixed by Manfred (thanks!).

However, we have an azure-sdk dependency that transitively pulls in
Guava (see https://issues.apache.org/jira/browse/OAK-10371).
Fortunately, there's a major update to the SDK which does not seem to
use Guava anymore.

Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 12.06.2023 15:50, Julian Reschke wrote:

...
We have made good progress, but as usual, things take longer than expected.

We have a pending PR for fixing oak-commons
(https://issues.apache.org/jira/browse/OAK-10247), but I could not apply
it yet (due to the current status of a certain downstream project).

The current plan is to cut 1.54.0 once this has been done (early July),
and then do the remaining changes in 1.56.0.
...


Champagne (sort of).

Oak itself is now independent of native Guava, but we still use test
tooling that pulls it in (see OAK-10367). Depending on how we resolve
this, we could then remove the remains from config and POMs.

Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 06.04.2023 13:56, Julian Reschke wrote:

Hi everybody,

as some have noticed, we have begun the migration away from Guava 15.0
to a *shaded* version of Guava (latest and greatest).

A new subproject has been added (oak-shaded-guava), which repackages
Guava 31 under the package name o.a.jackrabbit.guava. See
 for details and
discussion.

If your IDE acts weird (be it Eclipse or IntelliJ), it's because it does
not understand the shaded subproject. Sorry for that. While there's no
fix for the IDEs, the workaround is to disable oak-shaded-guava
(Intellij) or remove it (Eclipse) - after having built it locally with
Maven.

We may decide at a later point to work around this by moving this out of
the reactor pom.

What's next?

1) We have a few sub projects that currently expose Guava in APIs; these
APIs have been deprecated for long, and I'll remove those APIs *after*
the next release (1.52.0).

2) In the meantime, everybody can help by converting "their" sub
projects. (If you do so, please add a sub-task to
). This usually means
adding the dependency to the POM, and rewriting import statements. I
usually just run:

~~~
#!/bin/bash

find . -name "*.java" -exec sed
"s/^import\scom\.google\.common\.\(.*;\)/import
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
find . -name "*.java" -exec sed
"s/^import\sstatic\scom\.google\.common\.\(.*;\)/import static
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
~~~

WARNING: this is destructive. Only run it on a Git checkout with no
local changes, and do not do it inside your project root. You have been
warned.

After running this, you may find that a Guava 15 method actually has
been removed. Usually googling for the method signature will tell you
what happened.

For the modules where I already did the conversion, I also took the
opportunity to convert things to standard JDK 11 lingo where simple.
https://issues.apache.org/jira/browse/OAK-10180 /
https://github.com/apache/jackrabbit-oak/pull/890 is a good example for
that.

The currently planned timeline is:

1.52.0 (May?): release with shaded guava subproject and as many
subprojects converted as possible

1.54.0 (July?): remove deprecated APIs exposing native Guava

1.56.0: completely get rid of our Guava 15 dependency, only use the
shaded variant.

Feedback either here or in
 appreciated.


We have made good progress, but as usual, things take longer than expected.

We have a pending PR for fixing oak-commons
(https://issues.apache.org/jira/browse/OAK-10247), but I could not apply
it yet (due to the current status of a certain downstream project).

The current plan is to cut 1.54.0 once this has been done (early July),
and then do the remaining changes in 1.56.0.

Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 06.04.2023 13:56, Julian Reschke wrote:

Hi everybody,

as some have noticed, we have begun the migration away from Guava 15.0
to a *shaded* version of Guava (latest and greatest).

A new subproject has been added (oak-shaded-guava), which repackages
Guava 31 under the package name o.a.jackrabbit.guava. See
 for details and
discussion.

If your IDE acts weird (be it Eclipse or IntelliJ), it's because it does
not understand the shaded subproject. Sorry for that. While there's no
fix for the IDEs, the workaround is to disable oak-shaded-guava
(Intellij) or remove it (Eclipse) - after having built it locally with
Maven.

We may decide at a later point to work around this by moving this out of
the reactor pom.

What's next?

1) We have a few sub projects that currently expose Guava in APIs; these
APIs have been deprecated for long, and I'll remove those APIs *after*
the next release (1.52.0).

2) In the meantime, everybody can help by converting "their" sub
projects. (If you do so, please add a sub-task to
). This usually means
adding the dependency to the POM, and rewriting import statements. I
usually just run:

~~~
#!/bin/bash

find . -name "*.java" -exec sed
"s/^import\scom\.google\.common\.\(.*;\)/import
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
find . -name "*.java" -exec sed
"s/^import\sstatic\scom\.google\.common\.\(.*;\)/import static
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
~~~

WARNING: this is destructive. Only run it on a Git checkout with no
local changes, and do not do it inside your project root. You have been
warned.

After running this, you may find that a Guava 15 method actually has
been removed. Usually googling for the method signature will tell you
what happened.
...


In the meantime, I have converted a good number of modules.

Namely missing are components from blobs, lucene, and sement storage,
where I would *really* appreciate that developers closer to that code
than me would step in.

For next week I'm planning to cut Oak 1.52.0. The plan is to remove
public APIs using native Guava by 1.54.0, and completely switch to
shaded Guava for internal use by 1.56.0.

Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 06.04.2023 13:56, Julian Reschke wrote:

Hi everybody,

as some have noticed, we have begun the migration away from Guava 15.0
to a *shaded* version of Guava (latest and greatest).

A new subproject has been added (oak-shaded-guava), which repackages
Guava 31 under the package name o.a.jackrabbit.guava. See
 for details and
discussion.

If your IDE acts weird (be it Eclipse or IntelliJ), it's because it does
not understand the shaded subproject. Sorry for that. While there's no
fix for the IDEs, the workaround is to disable oak-shaded-guava
(Intellij) or remove it (Eclipse) - after having built it locally with
Maven.


As noticed by Angela, for IntelliJ this requires a relatively recent
version. The context menu item is in the "maven" view, BTW.


We may decide at a later point to work around this by moving this out of
the reactor pom.

What's next?

1) We have a few sub projects that currently expose Guava in APIs; these
APIs have been deprecated for long, and I'll remove those APIs *after*
the next release (1.52.0).

2) In the meantime, everybody can help by converting "their" sub
projects. (If you do so, please add a sub-task to
). This usually means
adding the dependency to the POM, and rewriting import statements. I
usually just run:

~~~
#!/bin/bash

find . -name "*.java" -exec sed
"s/^import\scom\.google\.common\.\(.*;\)/import
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
find . -name "*.java" -exec sed
"s/^import\sstatic\scom\.google\.common\.\(.*;\)/import static
org.apache.jackrabbit.guava.common.\1/g" -i {} ";"
~~~

WARNING: this is destructive. Only run it on a Git checkout with no
local changes, and do not do it inside your project root. You have been
warned.

After running this, you may find that a Guava 15 method actually has
been removed. Usually googling for the method signature will tell you
what happened.

For the modules where I already did the conversion, I also took the
opportunity to convert things to standard JDK 11 lingo where simple.
https://issues.apache.org/jira/browse/OAK-10180 /
https://github.com/apache/jackrabbit-oak/pull/890 is a good example for
that.


In the meantime, I changed my mind on this. I only do the minimal
changes needed; everything else should be done as a separate activity.

While continuing this work, I encountered cases where code uses methods
that have been removed or renamed in Guava. I'm not going to work on
these subprojects; instead I will open tickets and try to assign them to
people who are more familiar with that part of the code base (see
https://issues.apache.org/jira/browse/OAK-10197 for instance).


The currently planned timeline is:

1.52.0 (May?): release with shaded guava subproject and as many
subprojects converted as possible

1.54.0 (July?): remove deprecated APIs exposing native Guava

1.56.0: completely get rid of our Guava 15 dependency, only use the
shaded variant.

Feedback either here or in
 appreciated.

Julian


Best regards, Julian

Re: Guava Migration (yes, a must read)

[Julian Reschke]

On 06.04.2023 15:51, Marcel Reutegger wrote:

Hi Julian,

Thanks for the hard work so far. It’s really good to see progress on
this issue. I encourage everyone to contribute and migrate Oak
modules to the new shaded Guava module.

Is there a way to instruct the build to fail if someone is trying
to introduce a dependency to a shaded Guava package in an
Oak API?


That's a good idea; will do some research next week.