[jira] [Commented] (HBASE-24828) Unable to call HBaseContext From PySpark
[ https://issues.apache.org/jira/browse/HBASE-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842842#comment-17842842 ] Junegunn Choi commented on HBASE-24828: --- {{sc}} is a Python object, so you can't use it to create {{{}HBaseContext{}}}. {noformat} >>> type(sc) {noformat} Instead, you can pass the JVM object via its {{_jsc}} member ({{{}JavaSparkObject{}}}) like so: {code:java} hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc._jsc.sc(), conf, None) {code} > Unable to call HBaseContext From PySpark > > > Key: HBASE-24828 > URL: https://issues.apache.org/jira/browse/HBASE-24828 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Reporter: Manas >Priority: Minor > > I'm able to get the JavaHBaseContext Object From PySpark but not HBaseContext > {code:java} > temp = sc._jvm.org.apache.hadoop.hbase.HBaseConfiguration > conf = temp.create() > hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, conf){code} > Running the above Code gives me this error > > {code:java} > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > AttributeError Traceback (most recent call last) > in engine > > 1 hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, > conf) > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in _call_(self, > *args) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in (.0) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/protocol.py in > get_command_part(parameter, python_proxy_pool) > 296 command_part += ";" + interface > 297 else: > --> 298 command_part = REFERENCE_TYPE + parameter._get_object_id() > 299 > 300 command_part += "\n" > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] (HBASE-24828) Unable to call HBaseContext From PySpark
[ https://issues.apache.org/jira/browse/HBASE-24828 ] Junegunn Choi deleted comment on HBASE-24828: --- was (Author: junegunn): {{sc}} is a Python object so you can't directly use it to create {{HBaseContext}}. {noformat} Instead, you have to the reference to {code} {code} > Unable to call HBaseContext From PySpark > > > Key: HBASE-24828 > URL: https://issues.apache.org/jira/browse/HBASE-24828 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Reporter: Manas >Priority: Minor > > I'm able to get the JavaHBaseContext Object From PySpark but not HBaseContext > {code:java} > temp = sc._jvm.org.apache.hadoop.hbase.HBaseConfiguration > conf = temp.create() > hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, conf){code} > Running the above Code gives me this error > > {code:java} > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > AttributeError Traceback (most recent call last) > in engine > > 1 hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, > conf) > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in _call_(self, > *args) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in (.0) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/protocol.py in > get_command_part(parameter, python_proxy_pool) > 296 command_part += ";" + interface > 297 else: > --> 298 command_part = REFERENCE_TYPE + parameter._get_object_id() > 299 > 300 command_part += "\n" > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-24828) Unable to call HBaseContext From PySpark
[ https://issues.apache.org/jira/browse/HBASE-24828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842841#comment-17842841 ] Junegunn Choi commented on HBASE-24828: --- {{sc}} is a Python object so you can't directly use it to create {{HBaseContext}}. {noformat} Instead, you have to the reference to {code} {code} > Unable to call HBaseContext From PySpark > > > Key: HBASE-24828 > URL: https://issues.apache.org/jira/browse/HBASE-24828 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Reporter: Manas >Priority: Minor > > I'm able to get the JavaHBaseContext Object From PySpark but not HBaseContext > {code:java} > temp = sc._jvm.org.apache.hadoop.hbase.HBaseConfiguration > conf = temp.create() > hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, conf){code} > Running the above Code gives me this error > > {code:java} > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > AttributeError Traceback (most recent call last) > in engine > > 1 hbaseCon = sc._jvm.org.apache.hadoop.hbase.spark.HBaseContext(sc, > conf) > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in _call_(self, > *args) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/java_gateway.py in (.0) > 1543 > 1544 args_command = "".join( > -> 1545 [get_command_part(arg, self._pool) for arg in new_args]) > 1546 > 1547 command = proto.CONSTRUCTOR_COMMAND_NAME +\ > /usr/local/lib/python3.6/site-packages/py4j/protocol.py in > get_command_part(parameter, python_proxy_pool) > 296 command_part += ";" + interface > 297 else: > --> 298 command_part = REFERENCE_TYPE + parameter._get_object_id() > 299 > 300 command_part += "\n" > AttributeError: 'SparkContext' object has no attribute '_get_object_id' > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27118 Add security headers to Thrift/HTTP server [hbase]
stoty commented on code in PR #5864: URL: https://github.com/apache/hbase/pull/5864#discussion_r1587032854 ## hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java: ## @@ -387,9 +388,12 @@ protected void setupHTTPServer() throws IOException { httpServer = new Server(threadPool); // Context handler +boolean isSecure = conf.getBoolean(THRIFT_SSL_ENABLED_KEY, false); ServletContextHandler ctxHandler = new ServletContextHandler(httpServer, "/", ServletContextHandler.SESSIONS); -ctxHandler.addServlet(new ServletHolder(thriftHttpServlet), "/*"); +HttpServerUtil.addClickjackingPreventionFilter(ctxHandler, conf, PATH_SPEC_ANY); Review Comment: nit: You mention in the ticket description that we add this when "HTTP support is enabled". While I understand that this refers to the HTTP frontend as opposed to the binary, it can still cause misundestandings. I suggest updating the ticket text to "HTTP/HTTPS support is enabled" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28561) Add separate fields for column family and qualifier in REST message format
[ https://issues.apache.org/jira/browse/HBASE-28561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated HBASE-28561: Description: The current format uses the archaic column field, which requires extra processing and copying to encode/decode the CF and CQ at both the server and client side. We need to: - Add a version field to the requests, to be enabled by clients that support the new format - Add the new fields to the JSON, XML and protobuf formats, and logic to use them. This should be doable in a backwards-compatible manner, with the server falling back to the old format if it receives an unversioned request. was: The current format uses the archaic column field, which requires extra processing and copying at both the server and client side. We need to: - Add a version field to the requests, to be enabled by clients that support the new format - Add the new fields to the JSON, XML and protobuf formats, and logic to use them. This should be doable in a backwards-compatible manner, with the server falling back to the old format if it receives an unversioned request. > Add separate fields for column family and qualifier in REST message format > -- > > Key: HBASE-28561 > URL: https://issues.apache.org/jira/browse/HBASE-28561 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Priority: Major > > The current format uses the archaic column field, which requires extra > processing and copying to encode/decode the CF and CQ at both the server and > client side. > We need to: > - Add a version field to the requests, to be enabled by clients that support > the new format > - Add the new fields to the JSON, XML and protobuf formats, and logic to use > them. > This should be doable in a backwards-compatible manner, with the server > falling back to the old format if it receives an unversioned request. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28561) Add separate fields for column family and qualifier in REST message format
[ https://issues.apache.org/jira/browse/HBASE-28561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned HBASE-28561: --- Assignee: Istvan Toth > Add separate fields for column family and qualifier in REST message format > -- > > Key: HBASE-28561 > URL: https://issues.apache.org/jira/browse/HBASE-28561 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > > The current format uses the archaic column field, which requires extra > processing and copying to encode/decode the CF and CQ at both the server and > client side. > We need to: > - Add a version field to the requests, to be enabled by clients that support > the new format > - Add the new fields to the JSON, XML and protobuf formats, and logic to use > them. > This should be doable in a backwards-compatible manner, with the server > falling back to the old format if it receives an unversioned request. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28561) Add separate fields for column family and qualifier in REST message format
Istvan Toth created HBASE-28561: --- Summary: Add separate fields for column family and qualifier in REST message format Key: HBASE-28561 URL: https://issues.apache.org/jira/browse/HBASE-28561 Project: HBase Issue Type: Improvement Components: REST Reporter: Istvan Toth The current format uses the archaic column field, which requires extra processing and copying at both the server and client side. We need to: - Add a version field to the requests, to be enabled by clients that support the new format - Add the new fields to the JSON, XML and protobuf formats, and logic to use them. This should be doable in a backwards-compatible manner, with the server falling back to the old format if it receives an unversioned request. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path (#5771) [hbase]
Apache-HBase commented on PR #5812: URL: https://github.com/apache/hbase/pull/5812#issuecomment-2089439322 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 4m 20s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-3 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 37s | branch-3 passed | | +1 :green_heart: | compile | 2m 48s | branch-3 passed | | +1 :green_heart: | checkstyle | 0m 38s | branch-3 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 30s | branch-3 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 45s | the patch passed | | +1 :green_heart: | compile | 2m 39s | the patch passed | | +1 :green_heart: | javac | 2m 39s | the patch passed | | +1 :green_heart: | checkstyle | 0m 36s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 4m 57s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 0m 42s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 33m 15s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5812/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5812 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 139ef8019fb9 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-3 / 6ad7d9ea06 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5812/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path (#5771) [hbase]
Apache-HBase commented on PR #5813: URL: https://github.com/apache/hbase/pull/5813#issuecomment-2089437348 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 9s | branch-2 passed | | +1 :green_heart: | compile | 2m 39s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 34s | branch-2 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 27s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | the patch passed | | +1 :green_heart: | compile | 2m 40s | the patch passed | | +1 :green_heart: | javac | 2m 40s | the patch passed | | +1 :green_heart: | checkstyle | 0m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 44s | Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 30m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5813/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5813 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 939a67521a40 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 72b5c8f503 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 80 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5813/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28448 CompressionTest hangs when run over a Ozone ofs path (#5771) [hbase]
Apache-HBase commented on PR #5814: URL: https://github.com/apache/hbase/pull/5814#issuecomment-2089436388 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 40s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.6 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 4s | branch-2.6 passed | | +1 :green_heart: | compile | 2m 35s | branch-2.6 passed | | +1 :green_heart: | checkstyle | 0m 36s | branch-2.6 passed | | +1 :green_heart: | spotless | 0m 42s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | branch-2.6 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 2m 36s | the patch passed | | +1 :green_heart: | javac | 2m 36s | the patch passed | | +1 :green_heart: | checkstyle | 0m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 32s | Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 29s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 30m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5814/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5814 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux a62a1a859358 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.6 / 211d5fce9b | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 80 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5814/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28523) Use a single get call in REST multiget endpoint
[ https://issues.apache.org/jira/browse/HBASE-28523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842823#comment-17842823 ] Hudson commented on HBASE-28523: Results for branch branch-2.5 [build #521 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/520/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Use a single get call in REST multiget endpoint > --- > > Key: HBASE-28523 > URL: https://issues.apache.org/jira/browse/HBASE-28523 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: beginner, pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > The REST multiget endpoint currently issues a separate HBase GET operation > for each key. > Use the method that accepts a list of keys instead. > That should be faster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28533) Split procedure rollback can leave parent region state in SPLITTING after completion
[ https://issues.apache.org/jira/browse/HBASE-28533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842824#comment-17842824 ] Hudson commented on HBASE-28533: Results for branch branch-2.5 [build #521 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/520/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/521/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Split procedure rollback can leave parent region state in SPLITTING after > completion > > > Key: HBASE-28533 > URL: https://issues.apache.org/jira/browse/HBASE-28533 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.5.8, 3.0.0-beta-2 >Reporter: Daniel Roudnitsky >Assignee: Daniel Roudnitsky >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > Depending on where the split procedure fails, SplitTableRegionProcedure > rollback can leave the parent region's in memory RegionStateNode in SPLITTING > after rollback is complete, when the parent region is still online on the > assigned region server and according to meta. This leaves active HMaster > believing that the parent region is offline according to its RegionStates, > and causes subsequent procedures that require the region to be online like > merge/split/move to fail to start. One workaround is to restart active > HMaster to reset the in memory record of region states. > Two example scenarios where this can happen: > * If we get to SPLIT_TABLE_REGION_CLOSE_PARENT_REGION and the parent region > has a replica which is in transition, the unassign procedure in that step is > never created/rolled back and we are left with the parent region state in > splitting. > * If region quotas are enabled and a split is run for a region whose > namespace is at its maximum region quota limit we will fail in > SPLIT_TABLE_REGION_PRE_OPERATION with QuotaExceededException and we are left > with the parent region state in splitting > The region replica case is demonstrated in > TestSplitTableRegionProcedure.testRollbackForSplitTableRegionWithReplica. > To reproduce the region quota case in HBase shell: > {code:java} > > create_namespace 'test_ns', {'hbase.namespace.quota.maxregions'=> 2} > > create 'test_ns:test_table', 'f1', {NUMREGIONS => 2, SPLITALGO => > > 'UniformSplit'} > > region_a = > > region_b = > > split region_a, 'x' > # HMaster will report: > pid=405, state=ROLLEDBACK, > exception=org.apache.hadoop.hbase.quotas.QuotaExceededException via > master-split-regions:org.apache.hadoop.hbase.quotas.QuotaExceededException: > Region split not possible for : as quota limits are exceeded ; > SplitTableRegionProcedure table=test_ns:test_table, parent=... > > merge_region region_a, region_b > ERROR: org.apache.hadoop.hbase.exceptions.MergeRegionException: > org.apache.hadoop.hbase.client.DoNotRetryRegionException: is not > OPEN; state=SPLITTING > > stop_master # trigger hmaster failover > > merge_region region_a, region_b # merge now succeeds {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27118) Add security headers to Thrift/HTTP server
[ https://issues.apache.org/jira/browse/HBASE-27118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andor Molnar updated HBASE-27118: - Status: Patch Available (was: Open) > Add security headers to Thrift/HTTP server > --- > > Key: HBASE-27118 > URL: https://issues.apache.org/jira/browse/HBASE-27118 > Project: HBase > Issue Type: Improvement > Components: Thrift >Affects Versions: 3.0.0-alpha-2, 2.2.7, 2.1.10, 2.0.6 >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Labels: pull-request-available > > In order to further improve HBase exposed service for vulnerability scanners, > would be nice to implement additional security headers for the Thrift server > when HTTP transport is enabled. > Similarly to REST and Web UIs, related tickets are attached. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28523) Use a single get call in REST multiget endpoint
[ https://issues.apache.org/jira/browse/HBASE-28523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842771#comment-17842771 ] Hudson commented on HBASE-28523: Results for branch master [build #1065 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Use a single get call in REST multiget endpoint > --- > > Key: HBASE-28523 > URL: https://issues.apache.org/jira/browse/HBASE-28523 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: beginner, pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > The REST multiget endpoint currently issues a separate HBase GET operation > for each key. > Use the method that accepts a list of keys instead. > That should be faster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28533) Split procedure rollback can leave parent region state in SPLITTING after completion
[ https://issues.apache.org/jira/browse/HBASE-28533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842772#comment-17842772 ] Hudson commented on HBASE-28533: Results for branch master [build #1065 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1065/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Split procedure rollback can leave parent region state in SPLITTING after > completion > > > Key: HBASE-28533 > URL: https://issues.apache.org/jira/browse/HBASE-28533 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.5.8, 3.0.0-beta-2 >Reporter: Daniel Roudnitsky >Assignee: Daniel Roudnitsky >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > Depending on where the split procedure fails, SplitTableRegionProcedure > rollback can leave the parent region's in memory RegionStateNode in SPLITTING > after rollback is complete, when the parent region is still online on the > assigned region server and according to meta. This leaves active HMaster > believing that the parent region is offline according to its RegionStates, > and causes subsequent procedures that require the region to be online like > merge/split/move to fail to start. One workaround is to restart active > HMaster to reset the in memory record of region states. > Two example scenarios where this can happen: > * If we get to SPLIT_TABLE_REGION_CLOSE_PARENT_REGION and the parent region > has a replica which is in transition, the unassign procedure in that step is > never created/rolled back and we are left with the parent region state in > splitting. > * If region quotas are enabled and a split is run for a region whose > namespace is at its maximum region quota limit we will fail in > SPLIT_TABLE_REGION_PRE_OPERATION with QuotaExceededException and we are left > with the parent region state in splitting > The region replica case is demonstrated in > TestSplitTableRegionProcedure.testRollbackForSplitTableRegionWithReplica. > To reproduce the region quota case in HBase shell: > {code:java} > > create_namespace 'test_ns', {'hbase.namespace.quota.maxregions'=> 2} > > create 'test_ns:test_table', 'f1', {NUMREGIONS => 2, SPLITALGO => > > 'UniformSplit'} > > region_a = > > region_b = > > split region_a, 'x' > # HMaster will report: > pid=405, state=ROLLEDBACK, > exception=org.apache.hadoop.hbase.quotas.QuotaExceededException via > master-split-regions:org.apache.hadoop.hbase.quotas.QuotaExceededException: > Region split not possible for : as quota limits are exceeded ; > SplitTableRegionProcedure table=test_ns:test_table, parent=... > > merge_region region_a, region_b > ERROR: org.apache.hadoop.hbase.exceptions.MergeRegionException: > org.apache.hadoop.hbase.client.DoNotRetryRegionException: is not > OPEN; state=SPLITTING > > stop_master # trigger hmaster failover > > merge_region region_a, region_b # merge now succeeds {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28425 Allow specify cluster key without zookeeper in replication [hbase]
Apache-HBase commented on PR #5865: URL: https://github.com/apache/hbase/pull/5865#issuecomment-2089086369 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 45s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 2m 57s | master passed | | +1 :green_heart: | shadedjars | 6m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 58s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 22s | the patch passed | | +1 :green_heart: | compile | 2m 33s | the patch passed | | +1 :green_heart: | javac | 2m 33s | the patch passed | | +1 :green_heart: | shadedjars | 5m 47s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-client generated 1 new + 26 unchanged - 0 fixed = 27 total (was 26) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 48s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 44s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 35s | hbase-replication in the patch passed. | | -1 :x: | unit | 266m 2s | hbase-server in the patch failed. | | -1 :x: | unit | 17m 1s | hbase-mapreduce in the patch failed. | | +1 :green_heart: | unit | 2m 0s | hbase-testing-util in the patch passed. | | | | 326m 0s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5865 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8b446d56e6bf 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d66866f41 | | Default Java | Eclipse Adoptium-17.0.10+7 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk17-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/testReport/ | | Max. process+thread count | 5998 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-replication hbase-server hbase-mapreduce hbase-testing-util U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28425 Allow specify cluster key without zookeeper in replication [hbase]
Apache-HBase commented on PR #5865: URL: https://github.com/apache/hbase/pull/5865#issuecomment-2089081075 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 45s | master passed | | +1 :green_heart: | compile | 3m 22s | master passed | | +1 :green_heart: | shadedjars | 6m 18s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 10s | the patch passed | | +1 :green_heart: | compile | 2m 44s | the patch passed | | +1 :green_heart: | javac | 2m 44s | the patch passed | | +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 17s | hbase-client generated 1 new + 44 unchanged - 0 fixed = 45 total (was 44) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 20s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 37s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 35s | hbase-replication in the patch passed. | | -1 :x: | unit | 258m 23s | hbase-server in the patch failed. | | -1 :x: | unit | 21m 7s | hbase-mapreduce in the patch failed. | | +1 :green_heart: | unit | 2m 42s | hbase-testing-util in the patch passed. | | | | 322m 24s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5865 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d14a7d468686 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d66866f41 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk11-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/testReport/ | | Max. process+thread count | 4776 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-replication hbase-server hbase-mapreduce hbase-testing-util U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28533) Split procedure rollback can leave parent region state in SPLITTING after completion
[ https://issues.apache.org/jira/browse/HBASE-28533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842770#comment-17842770 ] Hudson commented on HBASE-28533: Results for branch branch-3 [build #197 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Split procedure rollback can leave parent region state in SPLITTING after > completion > > > Key: HBASE-28533 > URL: https://issues.apache.org/jira/browse/HBASE-28533 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.5.8, 3.0.0-beta-2 >Reporter: Daniel Roudnitsky >Assignee: Daniel Roudnitsky >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > Depending on where the split procedure fails, SplitTableRegionProcedure > rollback can leave the parent region's in memory RegionStateNode in SPLITTING > after rollback is complete, when the parent region is still online on the > assigned region server and according to meta. This leaves active HMaster > believing that the parent region is offline according to its RegionStates, > and causes subsequent procedures that require the region to be online like > merge/split/move to fail to start. One workaround is to restart active > HMaster to reset the in memory record of region states. > Two example scenarios where this can happen: > * If we get to SPLIT_TABLE_REGION_CLOSE_PARENT_REGION and the parent region > has a replica which is in transition, the unassign procedure in that step is > never created/rolled back and we are left with the parent region state in > splitting. > * If region quotas are enabled and a split is run for a region whose > namespace is at its maximum region quota limit we will fail in > SPLIT_TABLE_REGION_PRE_OPERATION with QuotaExceededException and we are left > with the parent region state in splitting > The region replica case is demonstrated in > TestSplitTableRegionProcedure.testRollbackForSplitTableRegionWithReplica. > To reproduce the region quota case in HBase shell: > {code:java} > > create_namespace 'test_ns', {'hbase.namespace.quota.maxregions'=> 2} > > create 'test_ns:test_table', 'f1', {NUMREGIONS => 2, SPLITALGO => > > 'UniformSplit'} > > region_a = > > region_b = > > split region_a, 'x' > # HMaster will report: > pid=405, state=ROLLEDBACK, > exception=org.apache.hadoop.hbase.quotas.QuotaExceededException via > master-split-regions:org.apache.hadoop.hbase.quotas.QuotaExceededException: > Region split not possible for : as quota limits are exceeded ; > SplitTableRegionProcedure table=test_ns:test_table, parent=... > > merge_region region_a, region_b > ERROR: org.apache.hadoop.hbase.exceptions.MergeRegionException: > org.apache.hadoop.hbase.client.DoNotRetryRegionException: is not > OPEN; state=SPLITTING > > stop_master # trigger hmaster failover > > merge_region region_a, region_b # merge now succeeds {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28523) Use a single get call in REST multiget endpoint
[ https://issues.apache.org/jira/browse/HBASE-28523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842769#comment-17842769 ] Hudson commented on HBASE-28523: Results for branch branch-3 [build #197 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/197/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Use a single get call in REST multiget endpoint > --- > > Key: HBASE-28523 > URL: https://issues.apache.org/jira/browse/HBASE-28523 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: beginner, pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > The REST multiget endpoint currently issues a separate HBase GET operation > for each key. > Use the method that accepts a list of keys instead. > That should be faster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28425 Allow specify cluster key without zookeeper in replication [hbase]
Apache-HBase commented on PR #5865: URL: https://github.com/apache/hbase/pull/5865#issuecomment-2089015802 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 2m 6s | master passed | | +1 :green_heart: | shadedjars | 5m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 34s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 29s | the patch passed | | +1 :green_heart: | compile | 2m 4s | the patch passed | | +1 :green_heart: | javac | 2m 4s | the patch passed | | +1 :green_heart: | shadedjars | 5m 7s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 15s | hbase-client generated 1 new + 30 unchanged - 0 fixed = 31 total (was 30) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 54s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 28s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 0m 33s | hbase-replication in the patch passed. | | -1 :x: | unit | 236m 50s | hbase-server in the patch failed. | | -1 :x: | unit | 15m 51s | hbase-mapreduce in the patch failed. | | +1 :green_heart: | unit | 3m 4s | hbase-testing-util in the patch passed. | | | | 289m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5865 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fe964c898d37 5.4.0-174-generic #193-Ubuntu SMP Thu Mar 7 14:29:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d66866f41 | | Default Java | Temurin-1.8.0_352-b08 | | javadoc | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk8-hadoop3-check/output/diff-javadoc-javadoc-hbase-client.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-mapreduce.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/testReport/ | | Max. process+thread count | 5092 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-replication hbase-server hbase-mapreduce hbase-testing-util U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-25972 Dual File Compaction [hbase]
kadirozde commented on PR #5545: URL: https://github.com/apache/hbase/pull/5545#issuecomment-2088988994 @Apache9, would you please review the latest commit and let me know if you have more comments for this PR? I also updated the design doc to reflect the changes based on your review comments. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28425 Allow specify cluster key without zookeeper in replication [hbase]
Apache-HBase commented on PR #5865: URL: https://github.com/apache/hbase/pull/5865#issuecomment-2088680853 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 6s | master passed | | +1 :green_heart: | compile | 5m 57s | master passed | | +1 :green_heart: | checkstyle | 1m 57s | master passed | | +1 :green_heart: | spotless | 0m 57s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 5m 2s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 33s | the patch passed | | +1 :green_heart: | compile | 6m 5s | the patch passed | | -0 :warning: | javac | 0m 48s | hbase-client generated 2 new + 102 unchanged - 0 fixed = 104 total (was 102) | | -0 :warning: | javac | 3m 15s | hbase-server generated 1 new + 194 unchanged - 1 fixed = 195 total (was 195) | | -0 :warning: | javac | 0m 24s | hbase-testing-util generated 1 new + 13 unchanged - 0 fixed = 14 total (was 13) | | +1 :green_heart: | checkstyle | 1m 49s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 6m 30s | Patch does not cause any errors with Hadoop 3.3.6. | | +1 :green_heart: | spotless | 1m 0s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 5m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 1m 0s | The patch does not generate ASF License warnings. | | | | 54m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5865 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 6c3094f6684d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 3d66866f41 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-client.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-server.txt | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/artifact/yetus-general-check/output/diff-compile-javac-hbase-testing-util.txt | | Max. process+thread count | 81 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-replication hbase-server hbase-mapreduce hbase-testing-util U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5865/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28425) Allow specify cluster key without zookeeper in replication
[ https://issues.apache.org/jira/browse/HBASE-28425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HBASE-28425: --- Labels: pull-request-available (was: ) > Allow specify cluster key without zookeeper in replication > -- > > Key: HBASE-28425 > URL: https://issues.apache.org/jira/browse/HBASE-28425 > Project: HBase > Issue Type: Improvement > Components: Replication, Zookeeper >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > > When reviewing the usage of zookeeper in HBase, I found out that, we still > rely on zookeeper when specifying the cluster key when setting up > replication. If we want to completely hide zookeeper from outside a cluster, > we should also remove this cluster key. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28521) Use standard ConnectionRegistry and Client API to get region server list in in replication
[ https://issues.apache.org/jira/browse/HBASE-28521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28521. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-3+. Thanks [~zghao] and [~andor]! > Use standard ConnectionRegistry and Client API to get region server list in > in replication > -- > > Key: HBASE-28521 > URL: https://issues.apache.org/jira/browse/HBASE-28521 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0-beta-2 > > > This is for allowing specify cluster key without zookeeper in replication > peer config. > For now, we will set a watcher on zookeeper for fetching the region server > list for the remote cluster, this means we must know the zookeeper address of > the remote cluster. This should be fixed as we do not want to leak the > zookeeper outside the cluster itself. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28558) Fix constructors for sub classes of Connection
[ https://issues.apache.org/jira/browse/HBASE-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28558. --- Fix Version/s: 2.7.0 3.0.0-beta-2 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2+. Thanks [~zghao] and [~GeorryHuang] for reviewing! > Fix constructors for sub classes of Connection > -- > > Key: HBASE-28558 > URL: https://issues.apache.org/jira/browse/HBASE-28558 > Project: HBase > Issue Type: Bug > Components: Client, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > Fix For: 2.7.0, 3.0.0-beta-2 > > > We still have some testing classes which implement Connection but do not have > constructors with ConnectionRegistry as parameter. > This lead to some test failures on branch-2.x, but we'd better also fix them > on master as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28555) ThriftConnection does not need ConnectionRegistry
[ https://issues.apache.org/jira/browse/HBASE-28555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28555. --- Fix Version/s: (was: 2.7.0) Resolution: Duplicate > ThriftConnection does not need ConnectionRegistry > - > > Key: HBASE-28555 > URL: https://issues.apache.org/jira/browse/HBASE-28555 > Project: HBase > Issue Type: Bug > Components: Client, Thrift >Reporter: Duo Zhang >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28521 Use standard ConnectionRegistry and Client API to get reg… [hbase]
Apache9 merged PR #5825: URL: https://github.com/apache/hbase/pull/5825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-28558) Fix constructors for sub classes of Connection
[ https://issues.apache.org/jira/browse/HBASE-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-28558 started by Duo Zhang. - > Fix constructors for sub classes of Connection > -- > > Key: HBASE-28558 > URL: https://issues.apache.org/jira/browse/HBASE-28558 > Project: HBase > Issue Type: Bug > Components: Client, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > > We still have some testing classes which implement Connection but do not have > constructors with ConnectionRegistry as parameter. > This lead to some test failures on branch-2.x, but we'd better also fix them > on master as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HBASE-28558) Fix constructors for sub classes of Connection
[ https://issues.apache.org/jira/browse/HBASE-28558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang reassigned HBASE-28558: - Assignee: Duo Zhang > Fix constructors for sub classes of Connection > -- > > Key: HBASE-28558 > URL: https://issues.apache.org/jira/browse/HBASE-28558 > Project: HBase > Issue Type: Bug > Components: Client, test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Labels: pull-request-available > > We still have some testing classes which implement Connection but do not have > constructors with ConnectionRegistry as parameter. > This lead to some test failures on branch-2.x, but we'd better also fix them > on master as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28558 Fix constructors for sub classes of Connection [hbase]
Apache9 merged PR #5861: URL: https://github.com/apache/hbase/pull/5861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28523) Use a single get call in REST multiget endpoint
[ https://issues.apache.org/jira/browse/HBASE-28523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842561#comment-17842561 ] Hudson commented on HBASE-28523: Results for branch branch-2 [build #1046 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (x) {color:red}-1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Use a single get call in REST multiget endpoint > --- > > Key: HBASE-28523 > URL: https://issues.apache.org/jira/browse/HBASE-28523 > Project: HBase > Issue Type: Improvement > Components: REST >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: beginner, pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > The REST multiget endpoint currently issues a separate HBase GET operation > for each key. > Use the method that accepts a list of keys instead. > That should be faster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28533) Split procedure rollback can leave parent region state in SPLITTING after completion
[ https://issues.apache.org/jira/browse/HBASE-28533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842562#comment-17842562 ] Hudson commented on HBASE-28533: Results for branch branch-2 [build #1046 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (x) {color:red}-1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/1046/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Split procedure rollback can leave parent region state in SPLITTING after > completion > > > Key: HBASE-28533 > URL: https://issues.apache.org/jira/browse/HBASE-28533 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.5.8, 3.0.0-beta-2 >Reporter: Daniel Roudnitsky >Assignee: Daniel Roudnitsky >Priority: Major > Labels: pull-request-available > Fix For: 2.4.18, 2.7.0, 3.0.0-beta-2, 2.6.1, 2.5.9 > > > Depending on where the split procedure fails, SplitTableRegionProcedure > rollback can leave the parent region's in memory RegionStateNode in SPLITTING > after rollback is complete, when the parent region is still online on the > assigned region server and according to meta. This leaves active HMaster > believing that the parent region is offline according to its RegionStates, > and causes subsequent procedures that require the region to be online like > merge/split/move to fail to start. One workaround is to restart active > HMaster to reset the in memory record of region states. > Two example scenarios where this can happen: > * If we get to SPLIT_TABLE_REGION_CLOSE_PARENT_REGION and the parent region > has a replica which is in transition, the unassign procedure in that step is > never created/rolled back and we are left with the parent region state in > splitting. > * If region quotas are enabled and a split is run for a region whose > namespace is at its maximum region quota limit we will fail in > SPLIT_TABLE_REGION_PRE_OPERATION with QuotaExceededException and we are left > with the parent region state in splitting > The region replica case is demonstrated in > TestSplitTableRegionProcedure.testRollbackForSplitTableRegionWithReplica. > To reproduce the region quota case in HBase shell: > {code:java} > > create_namespace 'test_ns', {'hbase.namespace.quota.maxregions'=> 2} > > create 'test_ns:test_table', 'f1', {NUMREGIONS => 2, SPLITALGO => > > 'UniformSplit'} > > region_a = > > region_b = > > split region_a, 'x' > # HMaster will report: > pid=405, state=ROLLEDBACK, > exception=org.apache.hadoop.hbase.quotas.QuotaExceededException via > master-split-regions:org.apache.hadoop.hbase.quotas.QuotaExceededException: > Region split not possible for : as quota limits are exceeded ; > SplitTableRegionProcedure table=test_ns:test_table, parent=... > > merge_region region_a, region_b > ERROR: org.apache.hadoop.hbase.exceptions.MergeRegionException: > org.apache.hadoop.hbase.client.DoNotRetryRegionException: is not > OPEN; state=SPLITTING > > stop_master # trigger hmaster failover > > merge_region region_a, region_b # merge now succeeds {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)