[GitHub] phoenix pull request #314: PHOENIX-4820
GitHub user comnetwork opened a pull request: https://github.com/apache/phoenix/pull/314 PHOENIX-4820 PHOENIX-4820 You can merge this pull request into a Git repository by running: $ git pull https://github.com/comnetwork/phoenix 4.x-HBase-1.3 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/314.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #314 commit ff779298e8c521c8a86d78ea7364d8eff8942c1a Author: chenglei Date: 2018-07-26T05:32:13Z PHOENIX-4820 ---
[jira] [Updated] (PHOENIX-4822) The configuration "phoenix.query.dateFormatTimeZone" does't work on the client
[ https://issues.apache.org/jira/browse/PHOENIX-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaanai Zhang updated PHOENIX-4822: -- Attachment: PHOENIX-4822.patch_v2 > The configuration "phoenix.query.dateFormatTimeZone" does't work on the client > -- > > Key: PHOENIX-4822 > URL: https://issues.apache.org/jira/browse/PHOENIX-4822 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.11.0, 4.12.0 >Reporter: Jaanai Zhang >Priority: Major > Attachments: PHOENIX-4822.patch, PHOENIX-4822.patch_v2 > > > when add configuration "phoenix.query.dateFormatTimeZone" into hbase-site.xml > or Properites of Connection, it can not work still uses time zone with GTM -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] phoenix pull request #306: PHOENIX-4797 file not found or file exist excepti...
Github user 492066199 closed the pull request at: https://github.com/apache/phoenix/pull/306 ---
[jira] [Assigned] (PHOENIX-4820) Optimize OrderBy for ClientAggregatePlan
[
https://issues.apache.org/jira/browse/PHOENIX-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei reassigned PHOENIX-4820:
-
Assignee: chenglei
> Optimize OrderBy for ClientAggregatePlan
>
>
> Key: PHOENIX-4820
> URL: https://issues.apache.org/jira/browse/PHOENIX-4820
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.14.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Attachments: PHOENIX-4820_v1.patch
>
>
> Given a table
> {code}
> create table test (
>pk1 varchar not null ,
>pk2 varchar not null,
>pk3 varchar not null,
> v1 varchar,
> v2 varchar,
> CONSTRAINT TEST_PK PRIMARY KEY (
> pk1,
> pk2,
> pk3 ))
> {code}
> for following sql :
> {code}
> select a.ak3
> from (select substr(pk1,1,1) ak1,substr(pk2,1,1) ak2,substr(pk3,1,1)
> ak3,substr(v1,1,1) av1,substr(v2,1,1) av2 from test order by pk2,pk3 limit
> 10) a group by a.ak3,a.av1 order by a.ak3,a.av1
> {code}
> Intuitively, the above OrderBy statement {{order by a.ak3,a.av1}} should be
> compiled out because it match the group by statement, but in fact it is not.
> The problem is caused by the {{QueryCompiler.compileSingleQuery}} and
> {{QueryCompiler.compileSingleFlatQuery}},for
> {{QueryCompiler.compileSingleQuery}} method,because the inner query has order
> by, so in line 520, local variable {{isInRowKeyOrder}} is false:
> {code}
> 519context.setCurrentTable(tableRef);
> 520boolean isInRowKeyOrder = innerPlan.getGroupBy() ==
> GroupBy.EMPTY_GROUP_BY && innerPlan.getOrderBy() == OrderBy.EMPTY_ORDER_BY;
> {code}
> In {{QueryCompiler.compileSingleFlatQuery}},when {{OrderByCompiler.compile}}
> method is invoked, the last parameter {{isInRowKeyOrder}} is false:
> {code}
> 562OrderBy orderBy = OrderByCompiler.compile(context, select,
> groupBy, limit, offset, projector,
> 563groupBy == GroupBy.EMPTY_GROUP_BY ?
> innerPlanTupleProjector : null, isInRowKeyOrder);
> {code}
> So in following line 156 for {{OrderByCompiler.compile}},even though the
> {{tracker.isOrderPreserving}} is true, the OrderBy statement could not be
> compiled out.
> {code}
> 156 if (isInRowKeyOrder && tracker.isOrderPreserving()) {
> {code}
> In my opinion, with GroupBy, in following line 563 for
> {{QueryCompiler.compileSingleFlatQuery}} method, when we call
> {{OrderByCompiler.compile}} method, we no need to conside the
> {{isInRowKeyOrder}}, just like the previous parameter {{tupleProjector}}
> does.
> {code}
> 562OrderBy orderBy = OrderByCompiler.compile(context, select,
> groupBy, limit, offset, projector,
> 563groupBy == GroupBy.EMPTY_GROUP_BY ?
> innerPlanTupleProjector : null, isInRowKeyOrder);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4820) Optimize OrderBy for ClientAggregatePlan
[
https://issues.apache.org/jira/browse/PHOENIX-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenglei updated PHOENIX-4820:
--
Attachment: PHOENIX-4820_v1.patch
> Optimize OrderBy for ClientAggregatePlan
>
>
> Key: PHOENIX-4820
> URL: https://issues.apache.org/jira/browse/PHOENIX-4820
> Project: Phoenix
> Issue Type: Improvement
>Affects Versions: 4.14.0
>Reporter: chenglei
>Assignee: chenglei
>Priority: Major
> Attachments: PHOENIX-4820_v1.patch
>
>
> Given a table
> {code}
> create table test (
>pk1 varchar not null ,
>pk2 varchar not null,
>pk3 varchar not null,
> v1 varchar,
> v2 varchar,
> CONSTRAINT TEST_PK PRIMARY KEY (
> pk1,
> pk2,
> pk3 ))
> {code}
> for following sql :
> {code}
> select a.ak3
> from (select substr(pk1,1,1) ak1,substr(pk2,1,1) ak2,substr(pk3,1,1)
> ak3,substr(v1,1,1) av1,substr(v2,1,1) av2 from test order by pk2,pk3 limit
> 10) a group by a.ak3,a.av1 order by a.ak3,a.av1
> {code}
> Intuitively, the above OrderBy statement {{order by a.ak3,a.av1}} should be
> compiled out because it match the group by statement, but in fact it is not.
> The problem is caused by the {{QueryCompiler.compileSingleQuery}} and
> {{QueryCompiler.compileSingleFlatQuery}},for
> {{QueryCompiler.compileSingleQuery}} method,because the inner query has order
> by, so in line 520, local variable {{isInRowKeyOrder}} is false:
> {code}
> 519context.setCurrentTable(tableRef);
> 520boolean isInRowKeyOrder = innerPlan.getGroupBy() ==
> GroupBy.EMPTY_GROUP_BY && innerPlan.getOrderBy() == OrderBy.EMPTY_ORDER_BY;
> {code}
> In {{QueryCompiler.compileSingleFlatQuery}},when {{OrderByCompiler.compile}}
> method is invoked, the last parameter {{isInRowKeyOrder}} is false:
> {code}
> 562OrderBy orderBy = OrderByCompiler.compile(context, select,
> groupBy, limit, offset, projector,
> 563groupBy == GroupBy.EMPTY_GROUP_BY ?
> innerPlanTupleProjector : null, isInRowKeyOrder);
> {code}
> So in following line 156 for {{OrderByCompiler.compile}},even though the
> {{tracker.isOrderPreserving}} is true, the OrderBy statement could not be
> compiled out.
> {code}
> 156 if (isInRowKeyOrder && tracker.isOrderPreserving()) {
> {code}
> In my opinion, with GroupBy, in following line 563 for
> {{QueryCompiler.compileSingleFlatQuery}} method, when we call
> {{OrderByCompiler.compile}} method, we no need to conside the
> {{isInRowKeyOrder}}, just like the previous parameter {{tupleProjector}}
> does.
> {code}
> 562OrderBy orderBy = OrderByCompiler.compile(context, select,
> groupBy, limit, offset, projector,
> 563groupBy == GroupBy.EMPTY_GROUP_BY ?
> innerPlanTupleProjector : null, isInRowKeyOrder);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4799) Write cells using checkAndMutate to prevent conflicting changes
[ https://issues.apache.org/jira/browse/PHOENIX-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-4799: Attachment: (was: PHOENIX-4799-4.x-HBase-1.3-v2.patch) > Write cells using checkAndMutate to prevent conflicting changes > --- > > Key: PHOENIX-4799 > URL: https://issues.apache.org/jira/browse/PHOENIX-4799 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.15.0, 5.1.0 >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-4799-v2.patch, PHOENIX-4799.patch > > > In order to prevent race conditions when multiple client try to mutate the > same column before sending the request to mutate the column to the server do > a checkAndMutate with the column name being added/dropped. Also: > 1. When a view is created do a checkAndMutate with the columns in the view > where clause. > 2. When an index on a view is created do a checkAndMutate with the indexed > columns. > > To prevent a race condition in the DROP TABLE CASCADE case, when a table is > dropped do a checkAndMutate with the rowkey of the base table name. If a view > is created it also does a checkAndMutate with the same rowkey. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4799) Write cells using checkAndMutate to prevent conflicting changes
[ https://issues.apache.org/jira/browse/PHOENIX-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-4799: Attachment: PHOENIX-4799-v2.patch > Write cells using checkAndMutate to prevent conflicting changes > --- > > Key: PHOENIX-4799 > URL: https://issues.apache.org/jira/browse/PHOENIX-4799 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.15.0, 5.1.0 >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-4799-v2.patch, PHOENIX-4799.patch > > > In order to prevent race conditions when multiple client try to mutate the > same column before sending the request to mutate the column to the server do > a checkAndMutate with the column name being added/dropped. Also: > 1. When a view is created do a checkAndMutate with the columns in the view > where clause. > 2. When an index on a view is created do a checkAndMutate with the indexed > columns. > > To prevent a race condition in the DROP TABLE CASCADE case, when a table is > dropped do a checkAndMutate with the rowkey of the base table name. If a view > is created it also does a checkAndMutate with the same rowkey. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4799) Write cells using checkAndMutate to prevent conflicting changes
[ https://issues.apache.org/jira/browse/PHOENIX-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-4799: Attachment: PHOENIX-4799-4.x-HBase-1.3-v2.patch > Write cells using checkAndMutate to prevent conflicting changes > --- > > Key: PHOENIX-4799 > URL: https://issues.apache.org/jira/browse/PHOENIX-4799 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.15.0, 5.1.0 >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva >Priority: Major > Attachments: PHOENIX-4799-4.x-HBase-1.3-v2.patch, PHOENIX-4799.patch > > > In order to prevent race conditions when multiple client try to mutate the > same column before sending the request to mutate the column to the server do > a checkAndMutate with the column name being added/dropped. Also: > 1. When a view is created do a checkAndMutate with the columns in the view > where clause. > 2. When an index on a view is created do a checkAndMutate with the indexed > columns. > > To prevent a race condition in the DROP TABLE CASCADE case, when a table is > dropped do a checkAndMutate with the rowkey of the base table name. If a view > is created it also does a checkAndMutate with the same rowkey. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] phoenix issue #313: PHOENIX-4799 Write cells using checkAndMutate to prevent...
Github user twdsilva commented on the issue: https://github.com/apache/phoenix/pull/313 We can't add/drop columns directly to an index on a view. Views with their own child views will write a mutex using the physical table. @ChinmaySKulkarni and @karanmehta93 thanks for the feedback. I have updated the PR, please take a look. ---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user geraldss commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205297065
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -135,17 +142,25 @@ public ResultIterator iterator(ParallelScanGrouper
scanGrouper, Scan scan) throw
aggResultIterator = new
ClientUngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators);
aggResultIterator = new
UngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
} else {
-if (!groupBy.isOrderPreserving()) {
-int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt(
-QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
-List keyExpressions =
groupBy.getKeyExpressions();
+List keyExpressions = groupBy.getKeyExpressions();
+if (groupBy.isOrderPreserving()) {
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+} else {
+int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt
+(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
List keyExpressionOrderBy =
Lists.newArrayListWithExpectedSize(keyExpressions.size());
for (Expression keyExpression : keyExpressions) {
keyExpressionOrderBy.add(new
OrderByExpression(keyExpression, false, true));
}
-iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+
+if (useHashAgg) {
+boolean sort = orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY;
+aggResultIterator = new
ClientHashAggregatingResultIterator(context, iterator, serverAggregators,
keyExpressions, sort);
--- End diff --
Same as for previous comment. The GROUP BY cannot produce a reverse sort.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user geraldss commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205296815
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -135,17 +142,25 @@ public ResultIterator iterator(ParallelScanGrouper
scanGrouper, Scan scan) throw
aggResultIterator = new
ClientUngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators);
aggResultIterator = new
UngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
} else {
-if (!groupBy.isOrderPreserving()) {
-int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt(
-QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
-List keyExpressions =
groupBy.getKeyExpressions();
+List keyExpressions = groupBy.getKeyExpressions();
+if (groupBy.isOrderPreserving()) {
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+} else {
+int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt
+(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
List keyExpressionOrderBy =
Lists.newArrayListWithExpectedSize(keyExpressions.size());
for (Expression keyExpression : keyExpressions) {
keyExpressionOrderBy.add(new
OrderByExpression(keyExpression, false, true));
}
-iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+
+if (useHashAgg) {
+boolean sort = orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY;
--- End diff --
The GROUP BY cannot cause REV_ROW_KEY_ORDER_BY, because the GROUP BY cannot
specific or produce descending keys. This is a pre-existing assumption + design
in SORT_MERGE_JOIN. The SORT_MERGE_JOIN creates its own forward sort, and its
Tracker reports a forward sort.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user geraldss commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205296403
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/ClientHashAggregateIT.java
---
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.QueryUtil;
+import org.junit.Test;
+
+public class ClientHashAggregateIT extends ParallelStatsDisabledIT {
--- End diff --
The current tests cover when forward sort is required. My comment below
addresses reverse sort.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user geraldss commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205296128
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -183,13 +198,15 @@ public ExplainPlan getExplainPlan() throws
SQLException {
if (where != null) {
planSteps.add("CLIENT FILTER BY " + where.toString());
}
-if (!groupBy.isEmpty()) {
-if (!groupBy.isOrderPreserving()) {
-planSteps.add("CLIENT SORTED BY " +
groupBy.getKeyExpressions().toString());
-}
+if (groupBy.isEmpty()) {
+planSteps.add("CLIENT AGGREGATE INTO SINGLE ROW");
+} else if (groupBy.isOrderPreserving()) {
planSteps.add("CLIENT AGGREGATE INTO DISTINCT ROWS BY " +
groupBy.getExpressions().toString());
+} else if (useHashAgg) {
+planSteps.add("CLIENT HASH AGGREGATE INTO DISTINCT ROWS BY " +
groupBy.getExpressions().toString());
--- End diff --
Done.
---
Re: "Releasing" versions in Jira?
This is done. On Wed, Jul 25, 2018 at 1:36 PM, Josh Elser wrote: > I'm noticing that all previous versions still show up as "unreleased" in > Jira version management. > > I'd like to move those which have been released to "Released". This will > prevent them from being used as a new fixVersion. > > Any complaints?
Re: [DISCUSS] 4.15 or move to 5.0?
Master is now 5.1.0-HBase-2.0-SNAPSHOT and what was previously master is now 4.x-HBase-1.4. On Wed, Jul 25, 2018 at 12:23 PM, Josh Elser wrote: > (realized I dropped this) > > Thanks for the reply, Andrew, on the 4.x releases. No complaints from me > about your request. > > Meanwhile, I'd like to move forward with the renaming of master -> > 4.x-HBase-1.4 and 5.x-HBase-2.0 -> master since there haven't been any > objections. Please speak up by EOD if there are concerns. > > On 2018/07/12 19:55:41, Andrew Purtell wrote: > With > my work hat on I would request the 4.x line continue for as long as >> >> HBase 1.x does. Probably it would be best if eventually there is only one >> active 4.x branch that follows the HBase 1.x minor release progression. So >> it would target 1.4 today, then 1.5, then 1.6 (and I'm not sure there will >> be more 1.x after that but who can say, releasing is an emergent community >> process) >> >> On Thu, Jul 12, 2018 at 7:44 AM Josh Elser wrote: >> >> > I remember a passing comment, but I forget the decision: what's up for >> > the 4.x release line post 4.14 and 5.0? >> > >> > Was 5.x going to move to master and then make 4.x branches for any more >> > 4.x releases? >> > >> > Related question I think I know the answer to: how much longer are we >> > going to continue work on the 4.x line? (as long as there are related >> > HBase releases, I imagine) >> > >> > - Josh >> > >> >> >> -- >> Best regards, >> Andrew >> >> Words like orphans lost among the crosstalk, meaning torn from truth's >> decrepit hands >>- A23, Crosstalk >> >
[jira] [Updated] (PHOENIX-3991) ROW_TIMESTAMP on TIMESTAMP column type throws ArrayOutOfBound when upserting without providing a value.
[
https://issues.apache.org/jira/browse/PHOENIX-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated PHOENIX-3991:
-
Attachment: PHOENIX-3991-1.patch
> ROW_TIMESTAMP on TIMESTAMP column type throws ArrayOutOfBound when upserting
> without providing a value.
> ---
>
> Key: PHOENIX-3991
> URL: https://issues.apache.org/jira/browse/PHOENIX-3991
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Eric Belanger
>Assignee: Sergey Soldatov
>Priority: Major
> Attachments: PHOENIX-3991-1.patch
>
>
> {code:sql}
> CREATE TABLE TEST (
> CREATED TIMESTAMP NOT NULL,
> ID CHAR(36) NOT NULL,
> DEFINITION VARCHAR,
> CONSTRAINT TEST_PK PRIMARY KEY (CREATED ROW_TIMESTAMP, ID)
> )
> -- WORKS
> UPSERT INTO TEST (CREATED, ID, DEFINITION) VALUES (NOW(), 'A', 'DEFINITION
> A');
> -- ArrayOutOfBoundException
> UPSERT INTO TEST (ID, DEFINITION) VALUES ('A', 'DEFINITION A');
> {code}
> Stack Trace:
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 8
> at
> org.apache.phoenix.execute.MutationState.getNewRowKeyWithRowTimestamp(MutationState.java:554)
> at
> org.apache.phoenix.execute.MutationState.generateMutations(MutationState.java:640)
> at
> org.apache.phoenix.execute.MutationState.addRowMutations(MutationState.java:572)
> at
> org.apache.phoenix.execute.MutationState.send(MutationState.java:1003)
> at
> org.apache.phoenix.execute.MutationState.send(MutationState.java:1469)
> at
> org.apache.phoenix.execute.MutationState.commit(MutationState.java:1301)
> at
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:539)
> at
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:536)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:536)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205232207
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -183,13 +198,15 @@ public ExplainPlan getExplainPlan() throws
SQLException {
if (where != null) {
planSteps.add("CLIENT FILTER BY " + where.toString());
}
-if (!groupBy.isEmpty()) {
-if (!groupBy.isOrderPreserving()) {
-planSteps.add("CLIENT SORTED BY " +
groupBy.getKeyExpressions().toString());
-}
+if (groupBy.isEmpty()) {
+planSteps.add("CLIENT AGGREGATE INTO SINGLE ROW");
+} else if (groupBy.isOrderPreserving()) {
planSteps.add("CLIENT AGGREGATE INTO DISTINCT ROWS BY " +
groupBy.getExpressions().toString());
+} else if (useHashAgg) {
+planSteps.add("CLIENT HASH AGGREGATE INTO DISTINCT ROWS BY " +
groupBy.getExpressions().toString());
--- End diff --
Add CLIENT SORTED BY line here if sorting required for hash aggregate.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205229353
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/ClientHashAggregateIT.java
---
@@ -0,0 +1,191 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.phoenix.end2end;
+
+import static org.apache.phoenix.util.TestUtil.TEST_PROPERTIES;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertTrue;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.Properties;
+
+import org.apache.phoenix.util.PropertiesUtil;
+import org.apache.phoenix.util.QueryUtil;
+import org.junit.Test;
+
+public class ClientHashAggregateIT extends ParallelStatsDisabledIT {
--- End diff --
Add tests for when sort (forward & reverse) required for hash aggregate.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205231380
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -135,17 +142,25 @@ public ResultIterator iterator(ParallelScanGrouper
scanGrouper, Scan scan) throw
aggResultIterator = new
ClientUngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators);
aggResultIterator = new
UngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
} else {
-if (!groupBy.isOrderPreserving()) {
-int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt(
-QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
-List keyExpressions =
groupBy.getKeyExpressions();
+List keyExpressions = groupBy.getKeyExpressions();
+if (groupBy.isOrderPreserving()) {
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+} else {
+int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt
+(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
List keyExpressionOrderBy =
Lists.newArrayListWithExpectedSize(keyExpressions.size());
for (Expression keyExpression : keyExpressions) {
keyExpressionOrderBy.add(new
OrderByExpression(keyExpression, false, true));
}
-iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+
+if (useHashAgg) {
+boolean sort = orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY;
+aggResultIterator = new
ClientHashAggregatingResultIterator(context, iterator, serverAggregators,
keyExpressions, sort);
--- End diff --
Youâll need to pass through if forward or reverse scan. You might just
pass through orderBy instead of the boolean. Better still would be to let the
code below insert an Ordering result iterator so you wouldnât need the sort
logic at all in your new iterator.
---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205230186
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -135,17 +142,25 @@ public ResultIterator iterator(ParallelScanGrouper
scanGrouper, Scan scan) throw
aggResultIterator = new
ClientUngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators);
aggResultIterator = new
UngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
} else {
-if (!groupBy.isOrderPreserving()) {
-int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt(
-QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
-List keyExpressions =
groupBy.getKeyExpressions();
+List keyExpressions = groupBy.getKeyExpressions();
+if (groupBy.isOrderPreserving()) {
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+} else {
+int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt
+(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
List keyExpressionOrderBy =
Lists.newArrayListWithExpectedSize(keyExpressions.size());
for (Expression keyExpression : keyExpressions) {
keyExpressionOrderBy.add(new
OrderByExpression(keyExpression, false, true));
}
-iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+
+if (useHashAgg) {
+boolean sort = orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY;
--- End diff --
Should be true if OrderBy.RVS_ROW_KEY_ORDER_BY too.
---
[jira] [Resolved] (PHOENIX-4817) Phoenix Tracing Web Application
[ https://issues.apache.org/jira/browse/PHOENIX-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved PHOENIX-4817. - Resolution: Fixed Fix Version/s: 5.1.0 4.15.0 I've gone ahead and merged your PR. Thanks for the contribution, Vitaly! > Phoenix Tracing Web Application > --- > > Key: PHOENIX-4817 > URL: https://issues.apache.org/jira/browse/PHOENIX-4817 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.0, 4.13.1 > Environment: maven 3+, JDK7 >Reporter: Vitaliy >Assignee: Vitaliy >Priority: Minor > Labels: patch > Fix For: 4.15.0, 5.1.0 > > > # Phoenix Tracing Web Application can not be started, because traceserver.py > does not contain required dependencies. > # The path to the folder webapp is incorrectly specified, so Phoenix Tracing > Web Application can not find this folder. > # Phoenix Tracing Web Application does not check null value where it is > needed, so the application can not load List, Dependency Tree and Features. > # Phoenix Tracing Web Application creates a query with the wrong table > column names. > # Phoenix Tracing Web Application has only the default tracing-table name > (SYSTEM.TRACING_STATS) and does not get tracing-table name via hbase-site.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PHOENIX-4817) Phoenix Tracing Web Application
[ https://issues.apache.org/jira/browse/PHOENIX-4817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned PHOENIX-4817: --- Assignee: Vitaliy > Phoenix Tracing Web Application > --- > > Key: PHOENIX-4817 > URL: https://issues.apache.org/jira/browse/PHOENIX-4817 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.0, 4.13.1 > Environment: maven 3+, JDK7 >Reporter: Vitaliy >Assignee: Vitaliy >Priority: Minor > Labels: patch > > # Phoenix Tracing Web Application can not be started, because traceserver.py > does not contain required dependencies. > # The path to the folder webapp is incorrectly specified, so Phoenix Tracing > Web Application can not find this folder. > # Phoenix Tracing Web Application does not check null value where it is > needed, so the application can not load List, Dependency Tree and Features. > # Phoenix Tracing Web Application creates a query with the wrong table > column names. > # Phoenix Tracing Web Application has only the default tracing-table name > (SYSTEM.TRACING_STATS) and does not get tracing-table name via hbase-site.xml. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] phoenix pull request #311: PHOENIX-4817 Fixed Phoenix Tracing Web Applicatio...
Github user asfgit closed the pull request at: https://github.com/apache/phoenix/pull/311 ---
[GitHub] phoenix pull request #308: Client-side hash aggregation
Github user geraldss commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/308#discussion_r205206944
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/execute/ClientAggregatePlan.java
---
@@ -135,17 +142,24 @@ public ResultIterator iterator(ParallelScanGrouper
scanGrouper, Scan scan) throw
aggResultIterator = new
ClientUngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators);
aggResultIterator = new
UngroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
} else {
-if (!groupBy.isOrderPreserving()) {
-int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt(
-QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
-List keyExpressions =
groupBy.getKeyExpressions();
+List keyExpressions = groupBy.getKeyExpressions();
+if (groupBy.isOrderPreserving()) {
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+} else {
+int thresholdBytes =
context.getConnection().getQueryServices().getProps().getInt
+(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB,
QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES);
List keyExpressionOrderBy =
Lists.newArrayListWithExpectedSize(keyExpressions.size());
for (Expression keyExpression : keyExpressions) {
keyExpressionOrderBy.add(new
OrderByExpression(keyExpression, false, true));
}
-iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+
+if (useHashAgg) {
+aggResultIterator = new
ClientHashAggregatingResultIterator(context, iterator, serverAggregators,
keyExpressions);
+} else {
+iterator = new OrderedResultIterator(iterator,
keyExpressionOrderBy, thresholdBytes, null, null,
projector.getEstimatedRowByteSize());
+aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, keyExpressions);
+}
}
-aggResultIterator = new
ClientGroupedAggregatingResultIterator(LookAheadResultIterator.wrap(iterator),
serverAggregators, groupBy.getKeyExpressions());
aggResultIterator = new
GroupedAggregatingResultIterator(LookAheadResultIterator.wrap(aggResultIterator),
clientAggregators);
}
--- End diff --
Hi @JamesRTaylor - please review. The sort is now done only when necessary.
Thanks.
---
[jira] [Assigned] (PHOENIX-3991) ROW_TIMESTAMP on TIMESTAMP column type throws ArrayOutOfBound when upserting without providing a value.
[
https://issues.apache.org/jira/browse/PHOENIX-3991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov reassigned PHOENIX-3991:
Assignee: Sergey Soldatov
> ROW_TIMESTAMP on TIMESTAMP column type throws ArrayOutOfBound when upserting
> without providing a value.
> ---
>
> Key: PHOENIX-3991
> URL: https://issues.apache.org/jira/browse/PHOENIX-3991
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Eric Belanger
>Assignee: Sergey Soldatov
>Priority: Major
>
> {code:sql}
> CREATE TABLE TEST (
> CREATED TIMESTAMP NOT NULL,
> ID CHAR(36) NOT NULL,
> DEFINITION VARCHAR,
> CONSTRAINT TEST_PK PRIMARY KEY (CREATED ROW_TIMESTAMP, ID)
> )
> -- WORKS
> UPSERT INTO TEST (CREATED, ID, DEFINITION) VALUES (NOW(), 'A', 'DEFINITION
> A');
> -- ArrayOutOfBoundException
> UPSERT INTO TEST (ID, DEFINITION) VALUES ('A', 'DEFINITION A');
> {code}
> Stack Trace:
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException: 8
> at
> org.apache.phoenix.execute.MutationState.getNewRowKeyWithRowTimestamp(MutationState.java:554)
> at
> org.apache.phoenix.execute.MutationState.generateMutations(MutationState.java:640)
> at
> org.apache.phoenix.execute.MutationState.addRowMutations(MutationState.java:572)
> at
> org.apache.phoenix.execute.MutationState.send(MutationState.java:1003)
> at
> org.apache.phoenix.execute.MutationState.send(MutationState.java:1469)
> at
> org.apache.phoenix.execute.MutationState.commit(MutationState.java:1301)
> at
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:539)
> at
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:536)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:536)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
"Releasing" versions in Jira?
I'm noticing that all previous versions still show up as "unreleased" in Jira version management. I'd like to move those which have been released to "Released". This will prevent them from being used as a new fixVersion. Any complaints?
[jira] [Updated] (PHOENIX-4797) file not found or file exist exception when create global index use -snapshot option
[
https://issues.apache.org/jira/browse/PHOENIX-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-4797:
Fix Version/s: 5.1.0
> file not found or file exist exception when create global index use -snapshot
> option
>
>
> Key: PHOENIX-4797
> URL: https://issues.apache.org/jira/browse/PHOENIX-4797
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
>Reporter: sailingYang
>Assignee: sailingYang
>Priority: Major
> Fix For: 4.15.0, 5.1.0
>
>
> when use indextool with -snapshot option and if the mapreduce create multi
> mapper.this will cause the hdfs file not found or hdfs file exist
> exception。finally the mapreduce task must be failed. because the mapper use
> the same restore work dir.
> {code:java}
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:186)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:578)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:249)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:171)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:814)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.init(TableSnapshotResultIterator.java:77)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.(TableSnapshotResultIterator.java:73)
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The
> specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:180)
> ... 15 more
> Caused by: java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.createRegionOnFileSystem(HRegionFileSystem.java:877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6252)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:205)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:173)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:170)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-28 15:01:55 70909 [main] INFO org.apache.hadoop.mapreduce.Job - Task
> Id : attempt_1530004808977_0011_m_01_0, Status : FAILED
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
>
[jira] [Resolved] (PHOENIX-4797) file not found or file exist exception when create global index use -snapshot option
[
https://issues.apache.org/jira/browse/PHOENIX-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karan Mehta resolved PHOENIX-4797.
--
Resolution: Fixed
> file not found or file exist exception when create global index use -snapshot
> option
>
>
> Key: PHOENIX-4797
> URL: https://issues.apache.org/jira/browse/PHOENIX-4797
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
>Reporter: sailingYang
>Assignee: sailingYang
>Priority: Major
> Fix For: 4.15.0
>
>
> when use indextool with -snapshot option and if the mapreduce create multi
> mapper.this will cause the hdfs file not found or hdfs file exist
> exception。finally the mapreduce task must be failed. because the mapper use
> the same restore work dir.
> {code:java}
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:186)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:578)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:249)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:171)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:814)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.init(TableSnapshotResultIterator.java:77)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.(TableSnapshotResultIterator.java:73)
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The
> specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:180)
> ... 15 more
> Caused by: java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.createRegionOnFileSystem(HRegionFileSystem.java:877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6252)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:205)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:173)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:170)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-28 15:01:55 70909 [main] INFO org.apache.hadoop.mapreduce.Job - Task
> Id : attempt_1530004808977_0011_m_01_0, Status : FAILED
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
>
Re: [DISCUSS] 4.15 or move to 5.0?
(realized I dropped this) Thanks for the reply, Andrew, on the 4.x releases. No complaints from me about your request. Meanwhile, I'd like to move forward with the renaming of master -> 4.x-HBase-1.4 and 5.x-HBase-2.0 -> master since there haven't been any objections. Please speak up by EOD if there are concerns. On 2018/07/12 19:55:41, Andrew Purtell wrote: > With my work hat on I would request the 4.x line continue for as long as HBase 1.x does. Probably it would be best if eventually there is only one active 4.x branch that follows the HBase 1.x minor release progression. So it would target 1.4 today, then 1.5, then 1.6 (and I'm not sure there will be more 1.x after that but who can say, releasing is an emergent community process) On Thu, Jul 12, 2018 at 7:44 AM Josh Elser wrote: > I remember a passing comment, but I forget the decision: what's up for > the 4.x release line post 4.14 and 5.0? > > Was 5.x going to move to master and then make 4.x branches for any more > 4.x releases? > > Related question I think I know the answer to: how much longer are we > going to continue work on the 4.x line? (as long as there are related > HBase releases, I imagine) > > - Josh > -- Best regards, Andrew Words like orphans lost among the crosstalk, meaning torn from truth's decrepit hands - A23, Crosstalk
Re: [DISCUSS] reduce notifications to phoenix-dev list
I think this would be a nice thing to do. On 7/24/18 8:17 PM, Thomas D'Silva wrote: Do folks think we need an issues mailing list similar to HBase that will receive notifications about comments? On Tue, Jul 24, 2018 at 5:10 PM, Thomas D'Silva wrote: Thanks Josh, I filed an INFRA Jira and updated the notification scheme. dev@phoenix.apache.org will no longer receive notifications when an issue is commented on or a comment is edited/deleted. On Thu, Jul 12, 2018 at 1:41 PM, Josh Elser wrote: I think INFRA keeps these locked down. Just requires an INFRA JIRA issue, I'd guess asking them to change it. On 7/12/18 1:53 PM, Thomas D'Silva wrote: Josh, Do you know how to edit the notification scheme here https://issues.apache.org/jira/plugins/servlet/project-confi g/PHOENIX/notifications ? I don't see an edit button. Thanks, Thomas On Thu, Jul 12, 2018 at 7:09 AM, James Taylor wrote: +1 On Thu, Jul 12, 2018 at 5:51 AM Josh Elser wrote: Fine by me. On 7/11/18 9:50 PM, Thomas D'Silva wrote: I think we should reduce the number of notifications that are sent to the phoenix-dev list. The notification scheme we use is located here : https://issues.apache.org/ jira/plugins/servlet/project-config/PHOENIX/notifications Maybe we don't need to notify the dev list when an issue is updated, or a comment is created/edited/deleted. People watching a particular issue would still see all the changes. This would be similar to how other projects like HBase and Calcite operate. Thanks, Thomas
[jira] [Reopened] (PHOENIX-4797) file not found or file exist exception when create global index use -snapshot option
[
https://issues.apache.org/jira/browse/PHOENIX-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser reopened PHOENIX-4797:
-
> file not found or file exist exception when create global index use -snapshot
> option
>
>
> Key: PHOENIX-4797
> URL: https://issues.apache.org/jira/browse/PHOENIX-4797
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
>Reporter: sailingYang
>Assignee: sailingYang
>Priority: Major
> Fix For: 4.15.0
>
>
> when use indextool with -snapshot option and if the mapreduce create multi
> mapper.this will cause the hdfs file not found or hdfs file exist
> exception。finally the mapreduce task must be failed. because the mapper use
> the same restore work dir.
> {code:java}
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:186)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:578)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:249)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:171)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:814)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.init(TableSnapshotResultIterator.java:77)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.(TableSnapshotResultIterator.java:73)
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The
> specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:180)
> ... 15 more
> Caused by: java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.createRegionOnFileSystem(HRegionFileSystem.java:877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6252)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:205)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:173)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:170)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-28 15:01:55 70909 [main] INFO org.apache.hadoop.mapreduce.Job - Task
> Id : attempt_1530004808977_0011_m_01_0, Status : FAILED
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
>
[jira] [Updated] (PHOENIX-3655) Metrics for PQS
[ https://issues.apache.org/jira/browse/PHOENIX-3655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karan Mehta updated PHOENIX-3655: - Attachment: PHOENIX-3655.001.diff > Metrics for PQS > --- > > Key: PHOENIX-3655 > URL: https://issues.apache.org/jira/browse/PHOENIX-3655 > Project: Phoenix > Issue Type: New Feature >Affects Versions: 4.8.0 > Environment: Linux 3.13.0-107-generic kernel, v4.9.0-HBase-0.98 >Reporter: Rahul Shrivastava >Assignee: Karan Mehta >Priority: Major > Fix For: 4.15.0 > > Attachments: MetricsforPhoenixQueryServerPQS.pdf, > PHOENIX-3655.001.diff > > Original Estimate: 240h > Remaining Estimate: 240h > > Phoenix Query Server runs a separate process compared to its thin client. > Metrics collection is currently done by PhoenixRuntime.java i.e. at Phoenix > driver level. We need the following > 1. For every jdbc statement/prepared statement/ run by PQS , we need > capability to collect metrics at PQS level and push the data to external sink > i.e. file, JMX , other external custom sources. > 2. Besides this global metrics could be periodically collected and pushed to > the sink. > 2. PQS can be configured to turn on metrics collection and type of collect ( > runtime or global) via hbase-site.xml > 3. Sink could be configured via an interface in hbase-site.xml. > All metrics definition https://phoenix.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4797) file not found or file exist exception when create global index use -snapshot option
[
https://issues.apache.org/jira/browse/PHOENIX-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-4797:
Fix Version/s: 4.15.0
> file not found or file exist exception when create global index use -snapshot
> option
>
>
> Key: PHOENIX-4797
> URL: https://issues.apache.org/jira/browse/PHOENIX-4797
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
>Reporter: sailingYang
>Assignee: sailingYang
>Priority: Major
> Fix For: 4.15.0
>
>
> when use indextool with -snapshot option and if the mapreduce create multi
> mapper.this will cause the hdfs file not found or hdfs file exist
> exception。finally the mapreduce task must be failed. because the mapper use
> the same restore work dir.
> {code:java}
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:186)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:578)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:249)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:171)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:814)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.init(TableSnapshotResultIterator.java:77)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.(TableSnapshotResultIterator.java:73)
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The
> specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:180)
> ... 15 more
> Caused by: java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.createRegionOnFileSystem(HRegionFileSystem.java:877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6252)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:205)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:173)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:170)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-28 15:01:55 70909 [main] INFO org.apache.hadoop.mapreduce.Job - Task
> Id : attempt_1530004808977_0011_m_01_0, Status : FAILED
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
>
[jira] [Assigned] (PHOENIX-4797) file not found or file exist exception when create global index use -snapshot option
[
https://issues.apache.org/jira/browse/PHOENIX-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser reassigned PHOENIX-4797:
---
Assignee: sailingYang
> file not found or file exist exception when create global index use -snapshot
> option
>
>
> Key: PHOENIX-4797
> URL: https://issues.apache.org/jira/browse/PHOENIX-4797
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.2-cdh5.11.2
>Reporter: sailingYang
>Assignee: sailingYang
>Priority: Major
>
> when use indextool with -snapshot option and if the mapreduce create multi
> mapper.this will cause the hdfs file not found or hdfs file exist
> exception。finally the mapreduce task must be failed. because the mapper use
> the same restore work dir.
> {code:java}
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:186)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.cloneHdfsRegions(RestoreSnapshotHelper.java:578)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:249)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.restoreHdfsRegions(RestoreSnapshotHelper.java:171)
> at
> org.apache.hadoop.hbase.snapshot.RestoreSnapshotHelper.copySnapshotForScanner(RestoreSnapshotHelper.java:814)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.init(TableSnapshotResultIterator.java:77)
> at
> org.apache.phoenix.iterate.TableSnapshotResultIterator.(TableSnapshotResultIterator.java:73)
> at
> org.apache.phoenix.mapreduce.PhoenixRecordReader.initialize(PhoenixRecordReader.java:126)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: The
> specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:188)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegions(ModifyRegionUtils.java:180)
> ... 15 more
> Caused by: java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.createRegionOnFileSystem(HRegionFileSystem.java:877)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:6252)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils.createRegion(ModifyRegionUtils.java:205)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:173)
> at
> org.apache.hadoop.hbase.util.ModifyRegionUtils$1.call(ModifyRegionUtils.java:170)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2018-06-28 15:01:55 70909 [main] INFO org.apache.hadoop.mapreduce.Job - Task
> Id : attempt_1530004808977_0011_m_01_0, Status : FAILED
> Error: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: The specified region already exists on disk:
> hdfs://m12v1.mlamp.cn:8020/tmp/index-snapshot-dir/restore-dir/e738c85b-2394-43fc-b9de-b8280bc329ca/data/default/SCOPA.CETUS_EVENT_ZY_SCOPA_31_0516_TRAIN_EVENT/2ab2c1d73d2e31bb5a5e2b394da564f8
> at
>
[GitHub] phoenix issue #311: PHOENIX-4817 Fixed Phoenix Tracing Web Application
Github user MrSandmanRUS commented on the issue: https://github.com/apache/phoenix/pull/311 Add to hbase-site.xml: phoenix.trace.enabled true ---
[GitHub] phoenix issue #311: PHOENIX-4817 Fixed Phoenix Tracing Web Application
Github user MrSandmanRUS commented on the issue: https://github.com/apache/phoenix/pull/311 Yes @mygood ---
[GitHub] phoenix issue #311: PHOENIX-4817 Fixed Phoenix Tracing Web Application
Github user mygood commented on the issue: https://github.com/apache/phoenix/pull/311 You use version from apache phoenix ãhbase ãhadoop @MrSandmanRUS ---
[jira] [Updated] (PHOENIX-4822) The configuration "phoenix.query.dateFormatTimeZone" does't work on the client
[ https://issues.apache.org/jira/browse/PHOENIX-4822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaanai Zhang updated PHOENIX-4822: -- Attachment: PHOENIX-4822.patch > The configuration "phoenix.query.dateFormatTimeZone" does't work on the client > -- > > Key: PHOENIX-4822 > URL: https://issues.apache.org/jira/browse/PHOENIX-4822 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.11.0, 4.12.0 >Reporter: Jaanai Zhang >Priority: Major > Attachments: PHOENIX-4822.patch > > > when add configuration "phoenix.query.dateFormatTimeZone" into hbase-site.xml > or Properites of Connection, it can not work still uses time zone with GTM -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4822) The configuration "phoenix.query.dateFormatTimeZone" does't work on the client
Jaanai Zhang created PHOENIX-4822: - Summary: The configuration "phoenix.query.dateFormatTimeZone" does't work on the client Key: PHOENIX-4822 URL: https://issues.apache.org/jira/browse/PHOENIX-4822 Project: Phoenix Issue Type: Bug Affects Versions: 4.11.0, 4.12.0 Reporter: Jaanai Zhang when add configuration "phoenix.query.dateFormatTimeZone" into hbase-site.xml or Properites of Connection, it can not work still uses time zone with GTM -- This message was sent by Atlassian JIRA (v7.6.3#76005)
