[jira] [Commented] (PHOENIX-4437) Make QueryPlan.getEstimatedBytesToScan() independent of getExplainPlan() and pull optimize() out of getExplainPlan()
[ https://issues.apache.org/jira/browse/PHOENIX-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369776#comment-16369776 ] Maryann Xue commented on PHOENIX-4437: -- Thank you, [~rajeshbabu], for reminding me! 5.x branch has now caught up with all my recent check-ins. > Make QueryPlan.getEstimatedBytesToScan() independent of getExplainPlan() and > pull optimize() out of getExplainPlan() > > > Key: PHOENIX-4437 > URL: https://issues.apache.org/jira/browse/PHOENIX-4437 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.11.0 >Reporter: Maryann Xue >Assignee: Maryann Xue >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4437.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-1556) Base hash versus sort merge join decision on cost
[ https://issues.apache.org/jira/browse/PHOENIX-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369777#comment-16369777 ] Maryann Xue commented on PHOENIX-1556: -- Thank you, [~rajeshbabu], for reminding me! 5.x branch has now caught up with all my recent check-ins. > Base hash versus sort merge join decision on cost > - > > Key: PHOENIX-1556 > URL: https://issues.apache.org/jira/browse/PHOENIX-1556 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maryann Xue >Priority: Major > Labels: CostBasedOptimization > Fix For: 4.14.0 > > Attachments: PHOENIX-1556.patch > > > At compile time, we know how many guideposts (i.e. how many bytes) will be > scanned for the RHS table. We should, by default, base the decision of using > the hash-join verus many-to-many join on this information. > Another criteria (as we've seen in PHOENIX-4508) is whether or not the tables > being joined are already ordered by the join key. In that case, it's better > to always use the sort merge join. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3050) Handle DESC columns in child/parent join optimization
[ https://issues.apache.org/jira/browse/PHOENIX-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369775#comment-16369775 ] Maryann Xue commented on PHOENIX-3050: -- Thank you, [~rajeshbabu], for reminding me! 5.x branch has now caught up with all my recent check-ins. > Handle DESC columns in child/parent join optimization > - > > Key: PHOENIX-3050 > URL: https://issues.apache.org/jira/browse/PHOENIX-3050 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Maryann Xue >Assignee: Maryann Xue >Priority: Minor > Fix For: 4.14.0 > > Attachments: PHOENIX-3050.patch > > > We found that child/parent join optimization would not work with DESC pk > columns. So as a quick fix for PHOENIX-3029, we simply avoid DESC columns > when optimizing, which would have no impact on the overall approach and no > impact on ASC columns. > > But eventually we need to make the optimization work with DESC columns too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4322) DESC primary key column with variable length does not work in SkipScanFilter
[
https://issues.apache.org/jira/browse/PHOENIX-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369774#comment-16369774
]
Maryann Xue commented on PHOENIX-4322:
--
Thank you, [~rajeshbabu], for reminding me! 5.x branch has now caught up with
all my recent check-ins.
> DESC primary key column with variable length does not work in SkipScanFilter
>
>
> Key: PHOENIX-4322
> URL: https://issues.apache.org/jira/browse/PHOENIX-4322
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>Priority: Minor
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4322.patch
>
>
> Example:
> {code}
> @Test
> public void inDescCompositePK3() throws Exception {
> String table = generateUniqueName();
> String ddl = "CREATE table " + table + " (oid VARCHAR NOT NULL, code
> VARCHAR NOT NULL constraint pk primary key (oid DESC, code DESC))";
> Object[][] insertedRows = new Object[][]{{"o1", "1"}, {"o2", "2"},
> {"o3", "3"}};
> runQueryTest(ddl, upsert("oid", "code"), insertedRows, new
> Object[][]{{"o2", "2"}, {"o1", "1"}}, new WhereCondition("(oid, code)", "IN",
> "(('o2', '2'), ('o1', '1'))"),
> table);
> }
> {code}
> Here the last column in primary key is in DESC order and has variable length,
> and WHERE clause involves an "IN" operator with RowValueConstructor
> specifying all PK columns. We get no results.
> This ends up being the root cause for not being able to use child/parent join
> optimization on DESC pk columns as described in PHOENIX-3050.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4288) Indexes not used when ordering by primary key
[ https://issues.apache.org/jira/browse/PHOENIX-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369773#comment-16369773 ] Maryann Xue commented on PHOENIX-4288: -- Thank you, [~rajeshbabu], for reminding me. 5.x branch has now caught up with all my recent check-ins. > Indexes not used when ordering by primary key > - > > Key: PHOENIX-4288 > URL: https://issues.apache.org/jira/browse/PHOENIX-4288 > Project: Phoenix > Issue Type: Sub-task >Reporter: Marcin Januszkiewicz >Assignee: Maryann Xue >Priority: Major > Labels: CostBasedOptimization > Fix For: 4.14.0 > > Attachments: PHOENIX-4288.patch > > > We have a table > CREATE TABLE t ( > rowkey VARCHAR PRIMARY KEY, > c1 VARCHAR, > c2 VARCHAR > ) > which we want to query by doing partial matches on c1, and keep the ordering > of the source table: > SELECT rowkey, c1, c2 FROM t where c1 LIKE 'X0%' ORDER BY rowkey; > We expect most queries to select a small subset of the table, so we create an > index to speed up searches: > CREATE LOCAL INDEX t_c1_ix ON t (c1); > However, this index will not be used since Phoenix will always choose not to > resort the data. > In our actual use case, adding index hints is not a practical solution. > See also discussion at: > https://lists.apache.org/thread.html/26ab58288eb811d2f074c3f89067163d341e5531fb581f3b2486cf43@%3Cuser.phoenix.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view
[
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4333:
--
Attachment: PHOENIX-4333_wip3.patch
> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
> Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch,
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch,
> PHOENIX-4333_wip3.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view
> yield partial results (only contains stats for B,1) which are incorrect even
> though it shows updated timestamp as current.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view
[
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4333:
--
Attachment: (was: PHOENIX-4333_wip3.patch)
> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
> Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch,
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch,
> PHOENIX-4333_wip3.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view
> yield partial results (only contains stats for B,1) which are incorrect even
> though it shows updated timestamp as current.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4231) Support restriction of remote UDF load sources
[
https://issues.apache.org/jira/browse/PHOENIX-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369703#comment-16369703
]
ASF GitHub Bot commented on PHOENIX-4231:
-
Github user aertoria commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/292#discussion_r169213703
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/UserDefinedFunctionsIT.java
---
@@ -322,6 +327,99 @@ public void testDeleteJar() throws Exception {
assertFalse(rs.next());
}
+/**
+ * Test adding jars from an HDFS URI.
+ * @throws Exception
+ */
+@Test
+public void testAddJarsFromHDFS() throws Exception {
+compileTestClass(MY_ARRAY_INDEX_CLASS_NAME,
MY_ARRAY_INDEX_PROGRAM, 7);
+Statement stmt = driver.connect(url,
EMPTY_PROPS).createStatement();
+// Note that we have already added all locally created UDF jars to
the hbase.dynamic.jars.dir directory
+ResultSet rs = stmt.executeQuery("list jars");
+int count = 0;
+while(rs.next()) {
+count++;
+}
+Path destJarPathOnHDFS =
copyJarsFromDynamicJarsDirToDummyHDFSDir("myjar7.jar");
+stmt.execute("delete jar '"+
util.getConfiguration().get(QueryServices.DYNAMIC_JARS_DIR_KEY)+"/"+"myjar7.jar'");
+stmt.execute("add jars '" + destJarPathOnHDFS.toString() + "'");
+rs = stmt.executeQuery("list jars");
+int finalCount = 0;
+while(rs.next()) {
+finalCount++;
+}
+assertEquals(count, finalCount);
--- End diff --
original `list jars` equal to `list jars`, shouldn't the newly added jar
make it plus 1? i.e.,
assertEquals(count+1, finalCount);
> Support restriction of remote UDF load sources
> ---
>
> Key: PHOENIX-4231
> URL: https://issues.apache.org/jira/browse/PHOENIX-4231
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Chinmay Kulkarni
>Priority: Major
>
> When allowUserDefinedFunctions is true, users can load UDFs remotely via a
> jar file from any HDFS filesystem reachable on the network. The setting
> hbase.dynamic.jars.dir can be used to restrict locations for jar loading but
> is only applied to jars loaded from the local filesystem. We should
> implement support for similar restriction via configuration for jars loaded
> via hdfs:// URIs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] phoenix pull request #292: PHOENIX-4231: Support restriction of remote UDF l...
Github user aertoria commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/292#discussion_r169213703
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/UserDefinedFunctionsIT.java
---
@@ -322,6 +327,99 @@ public void testDeleteJar() throws Exception {
assertFalse(rs.next());
}
+/**
+ * Test adding jars from an HDFS URI.
+ * @throws Exception
+ */
+@Test
+public void testAddJarsFromHDFS() throws Exception {
+compileTestClass(MY_ARRAY_INDEX_CLASS_NAME,
MY_ARRAY_INDEX_PROGRAM, 7);
+Statement stmt = driver.connect(url,
EMPTY_PROPS).createStatement();
+// Note that we have already added all locally created UDF jars to
the hbase.dynamic.jars.dir directory
+ResultSet rs = stmt.executeQuery("list jars");
+int count = 0;
+while(rs.next()) {
+count++;
+}
+Path destJarPathOnHDFS =
copyJarsFromDynamicJarsDirToDummyHDFSDir("myjar7.jar");
+stmt.execute("delete jar '"+
util.getConfiguration().get(QueryServices.DYNAMIC_JARS_DIR_KEY)+"/"+"myjar7.jar'");
+stmt.execute("add jars '" + destJarPathOnHDFS.toString() + "'");
+rs = stmt.executeQuery("list jars");
+int finalCount = 0;
+while(rs.next()) {
+finalCount++;
+}
+assertEquals(count, finalCount);
--- End diff --
original `list jars` equal to `list jars`, shouldn't the newly added jar
make it plus 1? i.e.,
assertEquals(count+1, finalCount);
---
[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view
[
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4333:
--
Attachment: PHOENIX-4333_wip3.patch
> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
> Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch,
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch,
> PHOENIX-4333_wip3.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view
> yield partial results (only contains stats for B,1) which are incorrect even
> though it shows updated timestamp as current.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view
[
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369624#comment-16369624
]
James Taylor commented on PHOENIX-4333:
---
wip3 adds tests. Still needs some code cleanup/simplification, but it's
functionally complete.
> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
> Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch,
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch,
> PHOENIX-4333_wip3.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view
> yield partial results (only contains stats for B,1) which are incorrect even
> though it shows updated timestamp as current.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4619) Process transactional updates to local index on server-side
[
https://issues.apache.org/jira/browse/PHOENIX-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4619:
--
Description:
For local indexes, we'll want to continue to process updates on the
server-side. After PHOENIX-4278, updates even for local indexes are occurring
on the client-side. The reason is that we know the updates to the index table
will be a local write and we can generate the write on the server side. Having
a separate RPC and sending the updates across the wire would be tremendously
inefficient. On top of that, we need the region boundary information which we
have already in the coprocessor, but would need to retrieve it on the client
side (with a likely race condition too if a split occurs after we retrieve it).
To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
can be use on the server-side as well. The main change will be to change this
method signature to pass through an IndexMaintainer instead of a PTable (which
isn't available on the server-side):
{code}
public List getIndexUpdates(final PTable table, PTable index,
List dataMutations) throws IOException, SQLException {
{code}
I think this can be changed to the following instead and be used both client
and server side:
{code}
public List getIndexUpdates(final IndexMaintainer maintainer,
byte[] dataTableName, List dataMutations) throws IOException,
SQLException {
{code}
We can tweak the code that makes PhoenixTransactionalIndexer a noop for clients
>= 4.14 to have it execute if the index is a local index. The one downside is
that if there's a mix of local and global indexes on the same table, the index
update calculation will be done twice. I think having a mix of index types
would be rare, though, and we should advise against it.
There's also this code in UngroupedAggregateRegionObserver which needs to be
updated to write shadow cells for Omid:
{code}
} else if (buildLocalIndex) {
for (IndexMaintainer maintainer : indexMaintainers)
{
if (!results.isEmpty()) {
result.getKey(ptr);
ValueGetter valueGetter =
maintainer.createGetterFromKeyValues(
ImmutableBytesPtr.copyBytesIfNecessary(ptr),
results);
Put put =
maintainer.buildUpdateMutation(kvBuilder,
valueGetter, ptr,
results.get(0).getTimestamp(),
env.getRegion().getRegionInfo().getStartKey(),
env.getRegion().getRegionInfo().getEndKey());
indexMutations.add(put);
}
}
result.setKeyValues(results);
{code}
This is the code that builds a local index initially (unlike the global index
code path which executes an UPSERT SELECT on the client side to do this initial
population).
was:
For local indexes, we'll want to continue to process updates on the
server-side. After PHOENIX-4278, updates even for local indexes are occurring
on the client-side. The reason is that we know the updates to the index table
will be a local write and we can generate the write on the server side. Having
a separate RPC and sending the updates across the wire would be tremendously
inefficient. On top of that, we need the region boundary information which we
have already in the coprocessor, but would need to retrieve it on the client
side (with a likely race condition too if a split occurs after we retrieve it).
To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
can be use on the server-side as well. The main change will be to change this
method signature to pass through an IndexMaintainer instead of a PTable (which
isn't available on the server-side):
{code}
public List getIndexUpdates(final PTable table, PTable index,
List dataMutations) throws IOException, SQLException {
{code}
I think this can be changed to the following instead and be used both client
and server side:
{code}
public List getIndexUpdates(final IndexMaintainer maintainer,
byte[] dataTableName, List dataMutations) throws IOException,
SQLException {
{code}
We can tweak the code that makes PhoenixTransactionalIndexer a noop for clients
>= 4.14 to have it execute if the index is a local index. The one downside is
that if there's a mix of local and global indexes on the same table, the index
update calculation will be done twice. I think having a mix of index types
would be rare, though, and we should advise against it.
> Process transactional updates to local index on
[jira] [Updated] (PHOENIX-4619) Process transactional updates to local index on server-side
[
https://issues.apache.org/jira/browse/PHOENIX-4619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4619:
--
Description:
For local indexes, we'll want to continue to process updates on the
server-side. After PHOENIX-4278, updates even for local indexes are occurring
on the client-side. The reason is that we know the updates to the index table
will be a local write and we can generate the write on the server side. Having
a separate RPC and sending the updates across the wire would be tremendously
inefficient. On top of that, we need the region boundary information which we
have already in the coprocessor, but would need to retrieve it on the client
side (with a likely race condition too if a split occurs after we retrieve it).
To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
can be use on the server-side as well. The main change will be to change this
method signature to pass through an IndexMaintainer instead of a PTable (which
isn't available on the server-side):
{code}
public List getIndexUpdates(final PTable table, PTable index,
List dataMutations) throws IOException, SQLException {
{code}
I think this can be changed to the following instead and be used both client
and server side:
{code}
public List getIndexUpdates(final IndexMaintainer maintainer,
byte[] dataTableName, List dataMutations) throws IOException,
SQLException {
{code}
We can tweak the code that makes PhoenixTransactionalIndexer a noop for clients
>= 4.14 to have it execute if the index is a local index. The one downside is
that if there's a mix of local and global indexes on the same table, the index
update calculation will be done twice. I think having a mix of index types
would be rare, though, and we should advise against it.
was:
For local indexes, we'll want to continue to process updates on the
server-side. After PHOENIX-4278, updates even for local indexes are occurring
on the client-side. The reason is that we know the updates to the index table
will be a local write and we can generate the write on the server side. Having
a separate RPC and sending the updates across the wire would be tremendously
inefficient. On top of that, we need the region boundary information which we
have already in the coprocessor, but would need to retrieve it on the client
side (with a likely race condition too if a split occurs after we retrieve it).
To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
can be use on the server-side as well. The main change will be to change this
method signature to pass through an IndexMaintainer instead of a PTable (which
isn't available on the server-side):
{code}
public List getIndexUpdates(final PTable table, PTable index,
List dataMutations) throws IOException, SQLException {
{code}
We can tweak the code that makes PhoenixTransactionalIndexer a noop for clients
>= 4.14 to have it execute if the index is a local index. The one downside is
that if there's a mix of local and global indexes on the same table, the index
update calculation will be done twice. I think having a mix of index types
would be rare, though, and we should advise against it.
> Process transactional updates to local index on server-side
> ---
>
> Key: PHOENIX-4619
> URL: https://issues.apache.org/jira/browse/PHOENIX-4619
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Priority: Major
>
> For local indexes, we'll want to continue to process updates on the
> server-side. After PHOENIX-4278, updates even for local indexes are occurring
> on the client-side. The reason is that we know the updates to the index table
> will be a local write and we can generate the write on the server side.
> Having a separate RPC and sending the updates across the wire would be
> tremendously inefficient. On top of that, we need the region boundary
> information which we have already in the coprocessor, but would need to
> retrieve it on the client side (with a likely race condition too if a split
> occurs after we retrieve it).
> To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
> can be use on the server-side as well. The main change will be to change this
> method signature to pass through an IndexMaintainer instead of a PTable
> (which isn't available on the server-side):
> {code}
> public List getIndexUpdates(final PTable table, PTable index,
> List dataMutations) throws IOException, SQLException {
> {code}
> I think this can be changed to the following instead and be used both client
> and server side:
> {code}
> public List getIndexUpdates(final IndexMaintainer maintainer,
> byte[] dataTableName, List dataMutations) throws IOException,
> SQLException {
> {code}
> We can
[jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid
[ https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369522#comment-16369522 ] James Taylor commented on PHOENIX-3860: --- You mean after PHOENIX-4278, right? For local indexes, we'll want to continue to process updates on the server-side - that's an oversight on my part. The reason is that we know the updates to the index table will be a local write and we can generate the write on the server side. Having a separate RPC and sending the updates across the wire would be tremendously inefficient. On top of that, we need the region boundary information which we have already in the coprocessor, but would need to retrieve it on the client side (with a likely race condition too if a split occurs after we retrieve it). I filed PHOENIX-4619 for this additional work. > Implement TAL functionality for Omid > > > Key: PHOENIX-3860 > URL: https://issues.apache.org/jira/browse/PHOENIX-3860 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > > Implement TAL functionality for Omid in order to be able to use Omid as > Phoenix's transaction processing engine. > To support the integration jira [OMID-82] was opened that encapsulates all > Omid related development for Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4619) Process transactional updates to local index on server-side
James Taylor created PHOENIX-4619:
-
Summary: Process transactional updates to local index on
server-side
Key: PHOENIX-4619
URL: https://issues.apache.org/jira/browse/PHOENIX-4619
Project: Phoenix
Issue Type: Bug
Reporter: James Taylor
For local indexes, we'll want to continue to process updates on the
server-side. After PHOENIX-4278, updates even for local indexes are occurring
on the client-side. The reason is that we know the updates to the index table
will be a local write and we can generate the write on the server side. Having
a separate RPC and sending the updates across the wire would be tremendously
inefficient. On top of that, we need the region boundary information which we
have already in the coprocessor, but would need to retrieve it on the client
side (with a likely race condition too if a split occurs after we retrieve it).
To fix this, we need to modify PhoenixTxnIndexMutationGenerator such that it
can be use on the server-side as well. The main change will be to change this
method signature to pass through an IndexMaintainer instead of a PTable (which
isn't available on the server-side):
{code}
public List getIndexUpdates(final PTable table, PTable index,
List dataMutations) throws IOException, SQLException {
{code}
We can tweak the code that makes PhoenixTransactionalIndexer a noop for clients
>= 4.14 to have it execute if the index is a local index. The one downside is
that if there's a mix of local and global indexes on the same table, the index
update calculation will be done twice. I think having a mix of index types
would be rare, though, and we should advise against it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-1267) Set scan.setSmall(true) when appropriate
[ https://issues.apache.org/jira/browse/PHOENIX-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369273#comment-16369273 ] James Taylor commented on PHOENIX-1267: --- Yes, this sounds good, [~abhishek.chouhan]. I think the tricky part is figuring out what the threshold should be. > Set scan.setSmall(true) when appropriate > > > Key: PHOENIX-1267 > URL: https://issues.apache.org/jira/browse/PHOENIX-1267 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Abhishek Singh Chouhan >Priority: Major > Labels: SFDC, newbie > Fix For: 4.14.0 > > Attachments: smallscan.patch, smallscan2.patch, smallscan3.patch > > > There's a nice optimization that has been in HBase for a while now to set a > scan as "small". This prevents extra RPC calls, I believe. We should add a > hint for queries that forces it to be set/not set, and make our best guess on > when it should default to true. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-1267) Set scan.setSmall(true) when appropriate
[
https://issues.apache.org/jira/browse/PHOENIX-1267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369141#comment-16369141
]
Abhishek Singh Chouhan commented on PHOENIX-1267:
-
Apologies for the delay, got a bit busy.
* Was thinking of having an attribute as suggested above, something like
"phoenix.query.smallScanThreshold" in QueryServices and its default of 100 in
QueryServicesOption
* In BaseQueryPlan.iterator(...) where we check for SMALL hint, also check for
pointlookup and its count (since we would already have the calculated
scanranges by this time in context), something like
{code:java}
if (statement.getHint().hasHint(Hint.SMALL) || (scanRanges.isPointLookup() &&
scanRanges.getPointLookupCount() < smallScanThreshold)) {
scan.setSmall(true);
}{code}
[~jamestaylor] [~vincentpoon] Please let me know if this sounds okay and i'll
put up a patch. Thanks!!
> Set scan.setSmall(true) when appropriate
>
>
> Key: PHOENIX-1267
> URL: https://issues.apache.org/jira/browse/PHOENIX-1267
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: Abhishek Singh Chouhan
>Priority: Major
> Labels: SFDC, newbie
> Fix For: 4.14.0
>
> Attachments: smallscan.patch, smallscan2.patch, smallscan3.patch
>
>
> There's a nice optimization that has been in HBase for a while now to set a
> scan as "small". This prevents extra RPC calls, I believe. We should add a
> hint for queries that forces it to be set/not set, and make our best guess on
> when it should default to true.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid
[ https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369116#comment-16369116 ] Ohad Shacham commented on PHOENIX-3860: --- Hi [~jamestaylor], after PHOENIX-4274 all the writes to the index in transactional mode are done at the client side using the TAL. I also saw that this is the case for creating a global index from a non-empty table. What about creating a local index from a non-empty table? As far as I could see these writes are not performed using the TAL. How could we force these write to use the TAL? Thx! > Implement TAL functionality for Omid > > > Key: PHOENIX-3860 > URL: https://issues.apache.org/jira/browse/PHOENIX-3860 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ohad Shacham >Assignee: Ohad Shacham >Priority: Major > > Implement TAL functionality for Omid in order to be able to use Omid as > Phoenix's transaction processing engine. > To support the integration jira [OMID-82] was opened that encapsulates all > Omid related development for Phoenix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4610) Converge 4.x and 5.x branches
[ https://issues.apache.org/jira/browse/PHOENIX-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369012#comment-16369012 ] Rajeshbabu Chintaguntla edited comment on PHOENIX-4610 at 2/19/18 11:48 AM: [~elserj] Here are the list of issues not committed to 5.x branch. Will check with the corresponding developers whether they can port it otherwise will take up and commit. PHOENIX-4278 PHOENIX-1556 PHOENIX-4130 PHOENIX-4548 PHOENIX-4528 PHOENIX-4523 PHOENIX-4414 PHOENIX-4487 PHOENIX-4488 PHOENIX-4437 PHOENIX-4424 PHOENIX-3837 PHOENIX-3050 PHOENIX-4322 PHOENIX-4288 PHOENIX-672 PHOENIX-4360 PHOENIX-4198 - Ankit is working on this. was (Author: rajeshbabu): [~elserj] Here are the list of issues not committed to 5.x branch. Will check with the corresponding developers whether they can port it otherwise will take up and commit. PHOENIX-4278 PHOENIX-1556 PHOENIX-4130 PHOENIX-4548 PHOENIX-4528 PHOENIX-4523 PHOENIX-4414 PHOENIX-4487 PHOENIX-4488 PHOENIX-4437 PHOENIX-4424 PHOENIX-3837 PHOENIX-3050 PHOENIX-4322 PHOENIX-4288 PHOENIX-672 PHOENIX-4360 PHOENIX-4198 - Ankit is working on this. PHOENIX-4322 > Converge 4.x and 5.x branches > - > > Key: PHOENIX-4610 > URL: https://issues.apache.org/jira/browse/PHOENIX-4610 > Project: Phoenix > Issue Type: Task >Reporter: Josh Elser >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 5.0.0 > > > We have a quite a few improvements which have landed on the 4.x branches > which have missed the 5.x branch due to its earlier instability. Rajeshbabu > volunteered offline to me to start this onerous task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4360) Prevent System.Catalog from splitting
[ https://issues.apache.org/jira/browse/PHOENIX-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369039#comment-16369039 ] Rajeshbabu Chintaguntla commented on PHOENIX-4360: -- [~lhofhansl] This is missing in 5.x branch. Can you please commit to the branch as well. Thanks. > Prevent System.Catalog from splitting > - > > Key: PHOENIX-4360 > URL: https://issues.apache.org/jira/browse/PHOENIX-4360 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Blocker > Fix For: 4.14.0 > > Attachments: 4360-v2-0.98.txt, 4360-v2.txt, 4360.txt > > > Just talked to [~jamestaylor]. > It turns out that currently System.Catalog is not prevented from splitting > generally, but does not allow splitting within a schema. > In the multi-tenant case that is not good enough. When System.Catalog splits > and a base table and view end up in different regions the following can > happen: > * DROP CASCADE no longer works for those views > * Adding/removing columns to/from the base table no longer works > Until PHOENIX-3534 is done we should simply prevent System.Catalog from > splitting. (just like HBase:meta) > [~apurtell] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-672) Add GRANT and REVOKE commands using HBase AccessController
[ https://issues.apache.org/jira/browse/PHOENIX-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369036#comment-16369036 ] Rajeshbabu Chintaguntla commented on PHOENIX-672: - [~karanmehta93] This is missing in 5.x branch. Can you please commit to the branch as well. Thanks. > Add GRANT and REVOKE commands using HBase AccessController > -- > > Key: PHOENIX-672 > URL: https://issues.apache.org/jira/browse/PHOENIX-672 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Karan Mehta >Priority: Major > Labels: namespaces, security > Fix For: 4.14.0 > > Attachments: PHOENIX-672.001.patch, PHOENIX-672.002.patch, > PHOENIX-672.003.patch > > > In HBase 0.98, cell-level security will be available. Take a look at > [this](https://communities.intel.com/community/datastack/blog/2013/10/29/hbase-cell-security) > excellent blog post by @apurtell. Once Phoenix works on 0.96, we should add > support for security to our SQL grammar. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4288) Indexes not used when ordering by primary key
[ https://issues.apache.org/jira/browse/PHOENIX-4288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369035#comment-16369035 ] Rajeshbabu Chintaguntla commented on PHOENIX-4288: -- [~maryannxue] This is missing in 5.x branch. Can you please commit to the branch as well. Thanks. > Indexes not used when ordering by primary key > - > > Key: PHOENIX-4288 > URL: https://issues.apache.org/jira/browse/PHOENIX-4288 > Project: Phoenix > Issue Type: Sub-task >Reporter: Marcin Januszkiewicz >Assignee: Maryann Xue >Priority: Major > Labels: CostBasedOptimization > Fix For: 4.14.0 > > Attachments: PHOENIX-4288.patch > > > We have a table > CREATE TABLE t ( > rowkey VARCHAR PRIMARY KEY, > c1 VARCHAR, > c2 VARCHAR > ) > which we want to query by doing partial matches on c1, and keep the ordering > of the source table: > SELECT rowkey, c1, c2 FROM t where c1 LIKE 'X0%' ORDER BY rowkey; > We expect most queries to select a small subset of the table, so we create an > index to speed up searches: > CREATE LOCAL INDEX t_c1_ix ON t (c1); > However, this index will not be used since Phoenix will always choose not to > resort the data. > In our actual use case, adding index hints is not a practical solution. > See also discussion at: > https://lists.apache.org/thread.html/26ab58288eb811d2f074c3f89067163d341e5531fb581f3b2486cf43@%3Cuser.phoenix.apache.org%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-3050) Handle DESC columns in child/parent join optimization
[ https://issues.apache.org/jira/browse/PHOENIX-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369032#comment-16369032 ] Rajeshbabu Chintaguntla commented on PHOENIX-3050: -- [~maryannxue] This is missing in 5.x branch. Can you please commit it to the branch as well. Thanks. > Handle DESC columns in child/parent join optimization > - > > Key: PHOENIX-3050 > URL: https://issues.apache.org/jira/browse/PHOENIX-3050 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Maryann Xue >Assignee: Maryann Xue >Priority: Minor > Fix For: 4.14.0 > > Attachments: PHOENIX-3050.patch > > > We found that child/parent join optimization would not work with DESC pk > columns. So as a quick fix for PHOENIX-3029, we simply avoid DESC columns > when optimizing, which would have no impact on the overall approach and no > impact on ASC columns. > > But eventually we need to make the optimization work with DESC columns too. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4322) DESC primary key column with variable length does not work in SkipScanFilter
[
https://issues.apache.org/jira/browse/PHOENIX-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369034#comment-16369034
]
Rajeshbabu Chintaguntla commented on PHOENIX-4322:
--
[~maryannxue] This is missing in 5.x branch. Can you please commit it to the
branch as well. Thanks.
> DESC primary key column with variable length does not work in SkipScanFilter
>
>
> Key: PHOENIX-4322
> URL: https://issues.apache.org/jira/browse/PHOENIX-4322
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Maryann Xue
>Assignee: Maryann Xue
>Priority: Minor
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4322.patch
>
>
> Example:
> {code}
> @Test
> public void inDescCompositePK3() throws Exception {
> String table = generateUniqueName();
> String ddl = "CREATE table " + table + " (oid VARCHAR NOT NULL, code
> VARCHAR NOT NULL constraint pk primary key (oid DESC, code DESC))";
> Object[][] insertedRows = new Object[][]{{"o1", "1"}, {"o2", "2"},
> {"o3", "3"}};
> runQueryTest(ddl, upsert("oid", "code"), insertedRows, new
> Object[][]{{"o2", "2"}, {"o1", "1"}}, new WhereCondition("(oid, code)", "IN",
> "(('o2', '2'), ('o1', '1'))"),
> table);
> }
> {code}
> Here the last column in primary key is in DESC order and has variable length,
> and WHERE clause involves an "IN" operator with RowValueConstructor
> specifying all PK columns. We get no results.
> This ends up being the root cause for not being able to use child/parent join
> optimization on DESC pk columns as described in PHOENIX-3050.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-3837) Unable to set property on an index with Alter statement
[
https://issues.apache.org/jira/browse/PHOENIX-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369031#comment-16369031
]
Rajeshbabu Chintaguntla commented on PHOENIX-3837:
--
[~jamestaylor] [~aertoria] If you are not working on this will take up and port
it. Thanks.
> Unable to set property on an index with Alter statement
> ---
>
> Key: PHOENIX-3837
> URL: https://issues.apache.org/jira/browse/PHOENIX-3837
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3837-Merged_With_5.x-2.0-and-Master.patch,
> PHOENIX-3837.patch
>
>
> {{ALTER INDEX IDX_T ON T SET GUIDE_POSTS_WIDTH=1}}
> {noformat}
> Error: ERROR 601 (42P00): Syntax error. Encountered "SET" at line 1, column
> 102. (state=42P00,code=601)
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00):
> Syntax error. Encountered "SET" at line 1, column 102.
> at
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1299)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4437) Make QueryPlan.getEstimatedBytesToScan() independent of getExplainPlan() and pull optimize() out of getExplainPlan()
[ https://issues.apache.org/jira/browse/PHOENIX-4437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369029#comment-16369029 ] Rajeshbabu Chintaguntla commented on PHOENIX-4437: -- [~maryannxue] This is missing in 5.x branch. Can you please commit it to the branch. Thanks. > Make QueryPlan.getEstimatedBytesToScan() independent of getExplainPlan() and > pull optimize() out of getExplainPlan() > > > Key: PHOENIX-4437 > URL: https://issues.apache.org/jira/browse/PHOENIX-4437 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.11.0 >Reporter: Maryann Xue >Assignee: Maryann Xue >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4437.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4488) Cache config parameters for MetaDataEndPointImpl during initialization
[
https://issues.apache.org/jira/browse/PHOENIX-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369028#comment-16369028
]
Rajeshbabu Chintaguntla commented on PHOENIX-4488:
--
[~jamestaylor] This is missing in 5.x branch. Can you please commit to the
branch as well. Thanks.
> Cache config parameters for MetaDataEndPointImpl during initialization
> --
>
> Key: PHOENIX-4488
> URL: https://issues.apache.org/jira/browse/PHOENIX-4488
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4488.patch
>
>
> For example, see this code (which is called often):
> {code}
> boolean blockWriteRebuildIndex =
> env.getConfiguration().getBoolean(QueryServices.INDEX_FAILURE_BLOCK_WRITE,
> QueryServicesOptions.DEFAULT_INDEX_FAILURE_BLOCK_WRITE);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (PHOENIX-4610) Converge 4.x and 5.x branches
[ https://issues.apache.org/jira/browse/PHOENIX-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369012#comment-16369012 ] Rajeshbabu Chintaguntla edited comment on PHOENIX-4610 at 2/19/18 11:36 AM: [~elserj] Here are the list of issues not committed to 5.x branch. Will check with the corresponding developers whether they can port it otherwise will take up and commit. PHOENIX-4278 PHOENIX-1556 PHOENIX-4130 PHOENIX-4548 PHOENIX-4528 PHOENIX-4523 PHOENIX-4414 PHOENIX-4487 PHOENIX-4488 PHOENIX-4437 PHOENIX-4424 PHOENIX-3837 PHOENIX-3050 PHOENIX-4322 PHOENIX-4288 PHOENIX-672 PHOENIX-4360 PHOENIX-4198 - Ankit is working on this. PHOENIX-4322 was (Author: rajeshbabu): [~elserj] Here are the list of issues not committed to 5.x branch. Will check with the corresponding developers whether they can port it otherwise will take up and commit. PHOENIX-4278 PHOENIX-1556 PHOENIX-4130 PHOENIX-4548 PHOENIX-4528 PHOENIX-4523 PHOENIX-4414 PHOENIX-4473 PHOENIX-4487 PHOENIX-4488 PHOENIX-4437 PHOENIX-4424 PHOENIX-3837 PHOENIX-3050 PHOENIX-4322 PHOENIX-4288 PHOENIX-672 PHOENIX-4360 PHOENIX-4198 - Ankit is working on this. PHOENIX-4322 > Converge 4.x and 5.x branches > - > > Key: PHOENIX-4610 > URL: https://issues.apache.org/jira/browse/PHOENIX-4610 > Project: Phoenix > Issue Type: Task >Reporter: Josh Elser >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 5.0.0 > > > We have a quite a few improvements which have landed on the 4.x branches > which have missed the 5.x branch due to its earlier instability. Rajeshbabu > volunteered offline to me to start this onerous task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4523) phoenix.schema.isNamespaceMappingEnabled problem
[
https://issues.apache.org/jira/browse/PHOENIX-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369024#comment-16369024
]
Rajeshbabu Chintaguntla commented on PHOENIX-4523:
--
[~karanmehta93] This is missing in 5.x branch. Can you please commit it to
branch as well. Thanks.
> phoenix.schema.isNamespaceMappingEnabled problem
>
>
> Key: PHOENIX-4523
> URL: https://issues.apache.org/jira/browse/PHOENIX-4523
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.1
>Reporter: Flavio Pompermaier
>Assignee: Karan Mehta
>Priority: Major
> Fix For: 5.0.0-alpha, 4.14.0, 4.13.2-cdh5.11.2
>
> Attachments: PHOENIX-4523.001.4.x-HBase-0.98.patch,
> PHOENIX-4523.001.patch
>
>
> I'm using Phoenix 4.13 for CDH 5.11.2 parcel and enabling schemas made my
> code unusable.
> I think that this is not a bug of the CDH release, but of all 4.13.x releases.
> I have many parallel Phoenix connections and I always get the following
> exception:
> {code:java}
> Caused by: java.sql.SQLException:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.TableExistsException):
> SYSTEM:MUTEX
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2492)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2384)
> at
> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
> at
> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2384)
> at
> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
> at
> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
> at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> at java.sql.DriverManager.getConnection(DriverManager.java:664)
> at java.sql.DriverManager.getConnection(DriverManager.java:270)
> {code}
> This is caused by the fact that all the times the SYSTEM tables are
> recreated, and this cannot be done simultaneously.
> Trying to debug the issue I found that in
> ConnectionQueryServicesImpl.createSysMutexTable() the call to
> getSystemTableNames() always return an empty array and the SYSTEM:MUTEX
> table is always recreated.
> This because getSystemTableNames() doesn't consider the case when system
> tables have namespace enabled. Right now that method tries to get all tables
> starting with *SYSTEM.\**, while it should try to get the list of *SYSTEM:\**
> tables..
> I hope this could get fixed very soon,
> Flavio
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4548) UpgradeUtil.mapChildViewsToNamespace does not handle multi-tenant views that have the same name.
[ https://issues.apache.org/jira/browse/PHOENIX-4548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369022#comment-16369022 ] Rajeshbabu Chintaguntla commented on PHOENIX-4548: -- [~tdsilva] This is missing in 5.x branch. Can you please commit to the branch as well. > UpgradeUtil.mapChildViewsToNamespace does not handle multi-tenant views that > have the same name. > > > Key: PHOENIX-4548 > URL: https://issues.apache.org/jira/browse/PHOENIX-4548 > Project: Phoenix > Issue Type: Bug >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4548.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes
[ https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369020#comment-16369020 ] Rajeshbabu Chintaguntla commented on PHOENIX-4130: -- [~vincentpoon] This is missing in 5.x-HBase-2.0 branch. Can you able to port and commit this to the branch as well? otherwise can take up and port it. Thanks > Avoid server retries for mutable indexes > > > Key: PHOENIX-4130 > URL: https://issues.apache.org/jira/browse/PHOENIX-4130 > Project: Phoenix > Issue Type: Improvement >Reporter: Lars Hofhansl >Assignee: Vincent Poon >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4130.addendum.master.patch, > PHOENIX-4130.v1.master.patch, PHOENIX-4130.v10.master.patch, > PHOENIX-4130.v2.master.patch, PHOENIX-4130.v3.master.patch, > PHOENIX-4130.v4.master.patch, PHOENIX-4130.v5.master.patch, > PHOENIX-4130.v6.master.patch, PHOENIX-4130.v7.master.patch, > PHOENIX-4130.v8.master.patch, PHOENIX-4130.v9.master.patch > > > Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], > during which I suggested that we can possibly eliminate retry loops happening > at the server that cause the handler threads to be stuck potentially for > quite a while (at least multiple seconds to ride over common scenarios like > splits). > Instead we can do the retries at the Phoenix client that. > So: > # The index updates are not retried on the server. (retries = 0) > # A failed index update would set the failed index timestamp but leave the > index enabled. > # Now the handler thread is done, it throws an appropriate exception back to > the client. > # The Phoenix client can now retry. When those retries fail the index is > disabled (if the policy dictates that) and throw the exception back to its > caller. > So no more waiting is needed on the server, handler threads are freed > immediately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-1556) Base hash versus sort merge join decision on cost
[ https://issues.apache.org/jira/browse/PHOENIX-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369018#comment-16369018 ] Rajeshbabu Chintaguntla commented on PHOENIX-1556: -- [~maryannxue] I think this is missing in 5.x-HBase-2.0 branch. Can you please commit this to the branch as well. Thanks. > Base hash versus sort merge join decision on cost > - > > Key: PHOENIX-1556 > URL: https://issues.apache.org/jira/browse/PHOENIX-1556 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Maryann Xue >Priority: Major > Labels: CostBasedOptimization > Fix For: 4.14.0 > > Attachments: PHOENIX-1556.patch > > > At compile time, we know how many guideposts (i.e. how many bytes) will be > scanned for the RHS table. We should, by default, base the decision of using > the hash-join verus many-to-many join on this information. > Another criteria (as we've seen in PHOENIX-4508) is whether or not the tables > being joined are already ordered by the join key. In that case, it's better > to always use the sort merge join. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4278) Implement pure client side transactional index maintenance
[ https://issues.apache.org/jira/browse/PHOENIX-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369014#comment-16369014 ] Rajeshbabu Chintaguntla commented on PHOENIX-4278: -- Working on this to port to 5.x. > Implement pure client side transactional index maintenance > -- > > Key: PHOENIX-4278 > URL: https://issues.apache.org/jira/browse/PHOENIX-4278 > Project: Phoenix > Issue Type: Improvement >Reporter: James Taylor >Assignee: Ohad Shacham >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4278.4.x-HBase-1.3.v1.patch, > PHOENIX-4278_v2.patch > > > The index maintenance for transactions follows the same model as non > transactional tables - coprocessor based on data table updates that looks up > previous row value to perform maintenance. This is necessary for non > transactional tables to ensure the rows are locked so that a consistent view > may be obtained. However, for transactional tables, the time stamp oracle > ensures uniqueness of time stamps (via transaction IDs) and the filtering > handles a scan seeing the "true" last committed value for a row. Thus, > there's no hard dependency to perform this on the server side. > Moving the index maintenance to the client side would prevent any RS->RS RPC > calls (which have proved to be troublesome for HBase). It would require > returning more data to the client (i.e. the prior row value), but this seems > like a reasonable tradeoff. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4610) Converge 4.x and 5.x branches
[ https://issues.apache.org/jira/browse/PHOENIX-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369012#comment-16369012 ] Rajeshbabu Chintaguntla commented on PHOENIX-4610: -- [~elserj] Here are the list of issues not committed to 5.x branch. Will check with the corresponding developers whether they can port it otherwise will take up and commit. PHOENIX-4278 PHOENIX-1556 PHOENIX-4130 PHOENIX-4548 PHOENIX-4528 PHOENIX-4523 PHOENIX-4414 PHOENIX-4473 PHOENIX-4487 PHOENIX-4488 PHOENIX-4437 PHOENIX-4424 PHOENIX-3837 PHOENIX-3050 PHOENIX-4322 PHOENIX-4288 PHOENIX-672 PHOENIX-4360 PHOENIX-4198 - Ankit is working on this. PHOENIX-4322 > Converge 4.x and 5.x branches > - > > Key: PHOENIX-4610 > URL: https://issues.apache.org/jira/browse/PHOENIX-4610 > Project: Phoenix > Issue Type: Task >Reporter: Josh Elser >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 5.0.0 > > > We have a quite a few improvements which have landed on the 4.x branches > which have missed the 5.x branch due to its earlier instability. Rajeshbabu > volunteered offline to me to start this onerous task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
