[jira] [Created] (HBASE-27340) Artifacts with resolved profiles
Michael Stack created HBASE-27340:
-
Summary: Artifacts with resolved profiles
Key: HBASE-27340
URL: https://issues.apache.org/jira/browse/HBASE-27340
Project: HBase
Issue Type: Brainstorming
Reporter: Michael Stack
Brainstorming/Discussion. The maven-flatten-plugin makes it so published poms
are 'flattened'. The poms contain the runtime-necessary dependencies only,
'build' and 'test' dependencies and plugins are dropped, versions are resolved
out of properties, and so on. The published poms are the barebones minimum
needed to run.
With a switch, the plugin can also make it so the produced poms have all
profiles 'resolved' – making it so the produced poms have all resolved hadoop2
or hadoop3 dependencies baked-in – based off which profile we used building.
(I've been interested in this flattening technique since I ran into a
downstreamer using hbase from a gradle build. Gradle does not respect profiles.
You can't specify that the gradle build pull in hbase with hadoop3 dependencies
using 'profiles'. I notice too our [~gjacoby] , [~apurtell] et al. up on the
dev list talking about making a hadoop3 set of artifacts...who might be
interested in this direction).
The attached patch adds the flatten plugin so folks can take a look-see. It
uncovers some locations where our versioning on dependencies is not explicit.
The workaround practiced here was adding hadoop2/hadoop3 profiles into
sub-modules that were missing them or moving problematic dependencies that were
outside of profiles under profiles in sub-modules that had them already. For
the latter, if the dependency specified excludes, the excludes were moved up to
the parent pom profile (parent pom profiles have dependencyManagement
sections... sub-modules have explicit dependency mentions... checks with
dependency:tree seem to show excludes continue to be effective).
This is the switch that flattens profiles:
true
This is the sort of complaint we had when the flatten plugin was having trouble
figure dependency versions – particularly hadoop versions
{{[ERROR] Failed to execute goal
org.codehaus.mojo:flatten-maven-plugin:1.3.0:flatten (flatten) on project
hbase-hadoop2-compat: 3 problems were encountered while building the effective
model for org.apache.hbase:hbase-hadoop2-compat:2.5.1-SNAPSHOT}}
{{[ERROR] [WARNING] 'build.plugins.plugin.version' for
org.codehaus.mojo:flatten-maven-plugin is missing. @}}
{{[ERROR] [ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-mapreduce-client-core:jar is missing. @}}
{{[ERROR] [ERROR] 'dependencies.dependency.version' for
javax.activation:javax.activation-api:jar is missing. @}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available
Seems this one might be rarely overriden, nothing problematic to have this change I think. Changing my vote to +1. On Fri, Aug 26, 2022 at 12:52 PM Viraj Jasani wrote: > +0 > > * Signature: ok > * Checksum : ok > * Rat check (1.8.0_341): ok > - mvn clean apache-rat:check > * Built from source (1.8.0_341): ok > - mvn clean install -DskipTests > * Unit tests pass (1.8.0_341): failed (TestUsersOperationsWithSecureHadoop > has known failure locally) > - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 > * A bit of high load testing locally looks good > > The compat report has few high sev problems reported but not all need to > be taken care of given that majority are affecting deprecated/private APIs. > However, there seems one high sev problem that we might want to consider > fixing: > > > org/apache/hadoop/hbase/io/encoding/DataBlockEncoding.createEncoder:(Ljava/lang/String;)Lorg/apache/hadoop/hbase/io/encoding/DataBlockEncoder > Access level has been changed from protected to package-private. > > This was done as part of the recent error-prone findings. Either we can > bring back protected access or, we already have public getter > *getEncoder()*, hence I think perhaps we can make *createEncoder(String > fullyQualifiedClassName)* as IA.Private. Either way is fine. > > Functional testing seems good, thanks Nick. > > > On Wed, Aug 24, 2022 at 2:53 AM Nick Dimiduk wrote: > >> Please vote on this Apache hbase release candidate, >> hbase-2.5.0RC1 >> >> The VOTE will remain open for at least 72 hours. >> >> [ ] +1 Release this package as Apache hbase 2.5.0 >> [ ] -1 Do not release this package because ... >> >> The tag to be voted on is 2.5.0RC1: >> >> https://github.com/apache/hbase/tree/2.5.0RC1 >> >> This tag currently points to git reference >> >> 2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab >> >> The release files, including signatures, digests, as well as CHANGES.md >> and RELEASENOTES.md included in this RC can be found at: >> >> https://dist.apache.org/repos/dist/dev/hbase/2.5.0RC1/ >> >> Maven artifacts are available in a staging repository at: >> >> https://repository.apache.org/content/repositories/orgapachehbase-1494/ >> >> Artifacts were signed with the 0x18567F39 key which can be found in: >> >> https://downloads.apache.org/hbase/KEYS >> >> To learn more about Apache hbase, please see >> >> http://hbase.apache.org/ >> >> Thanks, >> Your HBase Release Manager >> >
Re: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available
+1 (binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_202-ea): ok - mvn clean apache-rat:check * Built from source (1.8.0_202-ea): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_202-ea): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 From: Nick Dimiduk Date: Wednesday, August 24, 2022 at 11:53 AM To: hbase-dev Subject: [VOTE] Release candidate for HBase 2.5.0 (RC1) is available Please vote on this Apache hbase release candidate, hbase-2.5.0RC1 The VOTE will remain open for at least 72 hours. [ ] +1 Release this package as Apache hbase 2.5.0 [ ] -1 Do not release this package because ... The tag to be voted on is 2.5.0RC1: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhbase%2Ftree%2F2.5.0RC1data=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc664c2de261b4c7eed9708da85b679fa%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969316030503469%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=4nwVUe4NQqPy5jrCfza7rrtYFC%2F6ZZJ05OK6YpM%2FDsI%3Dreserved=0 This tag currently points to git reference 2ecd8bd6d615ca49bfb329b3c0c126c80846d4ab The release files, including signatures, digests, as well as CHANGES.md and RELEASENOTES.md included in this RC can be found at: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhbase%2F2.5.0RC1%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc664c2de261b4c7eed9708da85b679fa%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969316030503469%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=MyQGwxiUbrP56EuGi8ADzP3yekfilHw6ynTUJ4vrMWA%3Dreserved=0 Maven artifacts are available in a staging repository at: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachehbase-1494%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc664c2de261b4c7eed9708da85b679fa%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969316030503469%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=t1CCfiENCvcz4qX8sdvsHz%2FBzDuh%2BLj2domwlCVkImA%3Dreserved=0 Artifacts were signed with the 0x18567F39 key which can be found in: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fhbase%2FKEYSdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc664c2de261b4c7eed9708da85b679fa%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969316030503469%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=EyW%2BJWtUxDofIRXbfV2OaRurUVhFj4ZJmRWlOQe3IHw%3Dreserved=0 To learn more about Apache hbase, please see https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhbase.apache.org%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc664c2de261b4c7eed9708da85b679fa%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969316030503469%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=WBT0RnI5SrN4eLb3Z%2BD1GwTXe6Vm5B2YESYFQfjxcG0%3Dreserved=0 Thanks, Your HBase Release Manager
Re: [VOTE] HBase 2.4.14 release candidate (RC1) is available
+1 (binding) * Signature: ok * Checksum : ok * Rat check (1.8.0_202-ea): ok - mvn clean apache-rat:check * Built from source (1.8.0_202-ea): ok - mvn clean install -DskipTests * Unit tests pass (1.8.0_202-ea): ok - mvn package -P runSmallTests -Dsurefire.rerunFailingTestsCount=3 From: Huaxiang Sun Date: Wednesday, August 24, 2022 at 5:59 PM To: hbase-dev Subject: [VOTE] HBase 2.4.14 release candidate (RC1) is available [CAUTION] The sender of this email is from outside our organization. Please DO NOT CLICK any links, download attachments or respond unless you recognize the sender and know the content is safe. Please contact IT immediately in case you find it suspicious. Please vote on this Apache hbase release candidate, hbase-2.4.14RC1 The VOTE will remain open for at least 72 hours. [ ] +1 Release this package as Apache hbase 2.4.14 [ ] -1 Do not release this package because ... The tag to be voted on is 2.4.14RC1: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhbase%2Ftree%2F2.4.14RC1data=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=WXQ9FpS7wTYbTM%2B4qzjw7IFLTAjH51gMX%2BTDerymYFM%3Dreserved=0 This tag currently points to git reference 2e7d75a89271a7479b2f668c4db7a241be3f The release files, including signatures, digests, as well as CHANGES.md and RELEASENOTES.md included in this RC can be found at: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fhbase%2F2.4.14RC1%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=2LagAeb%2FCv0U2yktrTPFM5yNQOdaTJBHAoreUkvY7Uw%3Dreserved=0 Maven artifacts are available in a staging repository at: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachehbase-1495%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=gVKb99TUkHulfnYNIMgnIdllOV9t41%2Ffb5gyaQhQ%2F2U%3Dreserved=0 Artifacts were signed with the 0x117C835E key which can be found in: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdownloads.apache.org%2Fhbase%2FKEYSdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=HGrGgxPH6vlnHgVPJKIF7kP1eemT2f6aJ1UVaWp84%2Fk%3Dreserved=0 To learn more about Apache hbase, please see https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhbase.apache.org%2Fdata=05%7C01%7Cjan.hentschel%40ultratendency.com%7Cc0f1c6604c2441bf698208da85e9a4b3%7C8b2d44831cfd4f70b345dec07bd4f22f%7C0%7C0%7C637969535780563549%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7Csdata=R1nky03mcwjy%2FgYVOSl716Kch3PXYtJmhYARLIuAJz4%3Dreserved=0 Thanks, Your HBase Release Manager
[jira] [Resolved] (HBASE-27321) The ReplicationLogCleaner is not thread safe but can be called from different threads at the same time
[ https://issues.apache.org/jira/browse/HBASE-27321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-27321. --- Fix Version/s: 2.6.0 3.0.0-alpha-4 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. The change is a bit big so I do not port them to current release lines. Thanks [~sunxin] for reviewing! > The ReplicationLogCleaner is not thread safe but can be called from different > threads at the same time > -- > > Key: HBASE-27321 > URL: https://issues.apache.org/jira/browse/HBASE-27321 > Project: HBase > Issue Type: Bug > Components: master, Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Critical > Fix For: 2.6.0, 3.0.0-alpha-4 > > > In preClean method we will update the class fields and then use them in the > getDeletableFiles method. This implies that we will have only one cleaner run > at the same time. > But actually, in MasterRpcServices.runCleanerChore, we will call execute the > HFileCleaner and LogCleaner directly, not in the cleanerChore thread. > This could cause data loss for replication, which is a very serious problem. -- This message was sent by Atlassian Jira (v8.20.10#820010)
