[
https://issues.apache.org/jira/browse/HBASE-22114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807074#comment-16807074
]
Andrew Purtell edited comment on HBASE-22114 at 4/1/19 6:54 PM:
----------------------------------------------------------------
The checkstyle and whitespace warnings are legit, will fix those.
When using the make_rc.sh script the RAT check problem does not manifest. This
will not be a problem for RMs. Looks like precommit is invoking Maven build
targets in some different way that is problematic. (LIke running Java 8 and
Java 7 bits on the same precommit job without cleanup in between the steps.
That is a problem, wouldn't you say? Prevents build-with-jdk8 from really
separating the build logic, if precommit runs 7 and 8 in the same workspace
anyway.) Could we add a RAT exclusion like "***/target/***" ? Anyway seems like
a precommit only problem.
I also don't understand this:
{noformat}
[INFO] Apache HBase - TinyLFU Block Cache ................ SUCCESS [2.479s]
[INFO] Apache HBase - Integration Tests .................. FAILURE [0.202s]
...
[ERROR] Failed to execute goal on project hbase-it: Could not resolve
dependencies for project org.apache.hbase:hbase-it:jar:1.5.0: Failure to find
org.apache.hbase:hbase-tinylfu-blockcache:jar:1.5.0 in
https://repository.apache.org/content/repositories/releases/ was cached in the
local repository, resolution will not be reattempted until the update interval
of apache release has elapsed or updates are forced -> [Help 1]{noformat}
My guess is Yetus is not invoking the 'install' target here so the hbase-it
module has nothing in the local cache to resolve the dependency with. Doesn't
show up when you do the local build yourself. I always do 'mvn clean install
...'. I tested assembly with 'mvn clean install package assembly:single
-DskipTests' .
Also, it looks like javadoc was invoked on the tinylfu module under java 7 even
though that module is only ever conditionally defined and required from the
build-with-jdk8 profile, which has an activation predicate of jdk=1.8. Doesn't
happen when you build normally from the command line. I'm inclined to think
this is an issue with how Yetus decides what has changed and how Maven is
invoked to test those changes.
was (Author: apurtell):
The checkstyle and whitespace warnings are legit, will fix those.
When using the make_rc.sh script the RAT check problem does not manifest. This
will not be a problem for RMs. Looks like precommit is invoking Maven build
targets in some different way that is problematic. Could we add a RAT exclusion
like "***/target/***" ? Anyway seems like a precommit only problem.
I also don't understand this:
{noformat}
[INFO] Apache HBase - TinyLFU Block Cache ................ SUCCESS [2.479s]
[INFO] Apache HBase - Integration Tests .................. FAILURE [0.202s]
...
[ERROR] Failed to execute goal on project hbase-it: Could not resolve
dependencies for project org.apache.hbase:hbase-it:jar:1.5.0: Failure to find
org.apache.hbase:hbase-tinylfu-blockcache:jar:1.5.0 in
https://repository.apache.org/content/repositories/releases/ was cached in the
local repository, resolution will not be reattempted until the update interval
of apache release has elapsed or updates are forced -> [Help 1]{noformat}
My guess is Yetus is not invoking the 'install' target here so the hbase-it
module has nothing in the local cache to resolve the dependency with. Doesn't
show up when you do the local build yourself. I always do 'mvn clean install
...'. I tested assembly with 'mvn clean install package assembly:single
-DskipTests' .
Also, it looks like javadoc was invoked on the tinylfu module under java 7 even
though that module is only ever conditionally defined and required from the
build-with-jdk8 profile, which has an activation predicate of jdk=1.8. Doesn't
happen when you build normally from the command line. I'm inclined to think
this is an issue with how Yetus decides what has changed and how Maven is
invoked to test those changes.
> Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
> -------------------------------------------------------
>
> Key: HBASE-22114
> URL: https://issues.apache.org/jira/browse/HBASE-22114
> Project: HBase
> Issue Type: Sub-task
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Priority: Major
> Fix For: 1.5.0
>
> Attachments: HBASE-22114-branch-1.patch
>
>
> HBASE-15560 introduces the TinyLFU cache policy for the blockcache.
> W-TinyLFU ([research paper|http://arxiv.org/pdf/1512.00727.pdf]) records the
> frequency in a counting sketch, ages periodically by halving the counters,
> and orders entries by SLRU. An entry is discarded by comparing the frequency
> of the new arrival (candidate) to the SLRU's victim, and keeping the one with
> the highest frequency. This allows the operations to be performed in O(1)
> time and, though the use of a compact sketch, a much larger history is
> retained beyond the current working set. In a variety of real world traces
> the policy had [near optimal hit
> rates|https://github.com/ben-manes/caffeine/wiki/Efficiency].
> The implementation of HBASE-15560 uses several Java 8 idioms, depends on JRE
> 8+ type Optional, and has dependencies on libraries compiled with Java 8+
> bytecode. It could be backported to branch-1 but must be made optional both
> at compile time and runtime, enabled by the 'build-with-jdk8' build profile.
> The TinyLFU policy must go into its own build module.
> The blockcache must be modified to load L1 implementation/policy dynamically
> at startup by reflection if the policy is "TinyLFU"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
