Jacob Isaac created PHOENIX-7107:
Summary: Add support for indexing on SYSTEM.CATALOG table
Key: PHOENIX-7107
URL: https://issues.apache.org/jira/browse/PHOENIX-7107
Project: Phoenix
Issue
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni closed PHOENIX-3534.
-
Bulk closing Jiras for the 4.15.0 release.
> Support multi region SYSTEM.CATALOG ta
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated PHOENIX-3534:
--
Issue Type: Improvement (was: Bug)
> Support multi region SYSTEM.CATALOG ta
ion SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>
Assignee: Thomas D'Silva
>Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-3534-v2.patch, PHOENIX-3534-v3.patch,
> PHOENIX-3534.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region
> based
[
https://issues.apache.org/jira/browse/PHOENIX-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jaanai updated PHOENIX-4759:
Fix Version/s: 5.0.1
> During restart RS that hosts SYSTEM.CATALOG table may get st
.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>
t;crdBy" BIGINT,
"updAt" TIMESTAMP,
"updBy" BIGINT,
"stts"VARCHAR,
"lgcyId" VARCHAR,
CONSTRAINT "tracking_values_pk" PRIMARY KEY ("cstId", "cltId",
"
Could you reproduce this problem? What is your Pheonix's version number?
Jaanai Zhang
Best regards!
Ayola Jayamaha 于2018年11月28日周三 下午4:53写道：
> Hi,
> I have a question regarding system.catelog table
> How is this table is updated when we clone
Hi,
I have a question regarding system.catelog table
How is this table is updated when we clone hbase table and apply ddl in
pheonix client?
We see a problem when we clone the table. For example a table has 15 rows
in system.catalog we cloned the hbase snapshot and recreated the same
table
when
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva reopened PHOENIX-3534:
-
> Support multi region SYSTEM.CATALOG ta
ion SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/165/])
PHOENIX-3534 Support multi region SYSTEM.CATALOG table (Thomas D'Silva
(tdsilva: rev 93fdd5bad22cde313c9f34fe7448dca44377a27c)
* (add)
phoenix-core/src/it/java/org/apache/phoenix/end2end/SplitSystemCatalogIT.java
* (edit)
phoenix-core
at:
https://github.com/apache/phoenix/pull/303
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
>
propagateToViews) {
--- End diff --
I filed PHOENIX-4763 to fix this. We should be able to use the cell
timestamp to differentiate, still need to figure out how to expose this since
its the properties in PTable don't currently expose the timestamp.
> Support multi region SYSTEM.CATALOG ta
:
https://github.com/apache/phoenix/pull/303
@JamesRTaylor Thanks for the feedback, I have updated the PR. I will get
this committed shortly.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
>
Github user twdsilva commented on the issue:
https://github.com/apache/phoenix/pull/303
@JamesRTaylor Thanks for the feedback, I have updated the PR. I will get
this committed shortly.
---
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1926//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1926//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1926//c
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva updated PHOENIX-3534:
Attachment: PHOENIX-3534-v3.patch
> Support multi region SYSTEM.CATALOG ta
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hadoop QA updated PHOENIX-3534:
---
Attachment: PHOENIX-3534-v2.patch
> Support multi region SYSTEM.CATALOG ta
ata atomically.
We ignore views that can't be found (in case writing the child view
metadata fails).
If the metadata write fails and the table uses column encoding then we will
lose a few column qualifiers.
I'll add a test for this.
> Support multi r
t; Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>Reporter
iff --
I filed PHOENIX-4810 and added a comment to reference this jira.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
>
and then dropped
+// and clean up any parent->child links or child views
--- End diff --
Done.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issu
if (column.equals(SaltingUtil.SALTING_COLUMN) ||
column.isExcluded()) {
--- End diff --
I will change this.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
>
if (numOfChildViews > 0 && !allViewsInCurrentRegion) {
-tableViewFinderResult.setAllViewsNotInSingleRegion();
-}
-return tableViewFinderResult;
+
+private void findAncestorViewsOfIndex(byte[] tenantId, byte[]
schemaName, byte[] indexName,
--- En
);
}
-
+
--- End diff --
I modified the class level comment.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/brow
tadata(schemaName, tableName,
tenantIdBytes);
+ }
--- End diff --
Fixed.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jir
to the physical table. So the
+* viewPhysicalTableRow is null in that case.
+*/
--- End diff --
Fixed.
> Support multi region SYSTEM.CATALOG table
> -
>
>
} else if (!pColumn.equals(SaltingUtil.SALTING_COLUMN)) {
--- End diff --
I will change this.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https:
();
+}
--- End diff --
I created PHOENIX-4766 for this, I will add a comment referencing this JIRA
in createTableInternal().
> Support multi region SYSTEM.CATALOG table
> -
>
> Key:
to the base table and a
child view will be covered in PHOENIX-4799.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project:
I think we can handle the race condition in a similar way to how we handle
conflicting columns using checkAndMutate. I have updated PHOENIX-4799 to
include this case.
> Support multi region SYSTEM.CATALOG table
> -
>
>
(removing them from
SYSTEM.CATALOG and recreating them in the new SYSTEM.CHILD_LINK table). The
PHYSICAL_TABLE link overwriting the PARENT_TABLE link happens in the
CHILD->PARENT links.
> Support multi region SYSTEM.CATALOG table
> -
>
>
, but we should remember to do it.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
>
, KeyValue.COMPARATOR);
}
-
+
--- End diff --
Might be good to include a class level comment that explains the overall
approach at a high level.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key:
a first level child, then we need to
create the PARENT_TABLE link
+// that was overwritten by the PHYSICAL_TABLE link
--- End diff --
Ah, good. So we'll be consistent with the parent link now, right?
> Support multi regi
Isn't there a race condition with this check?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
// When namespace mapping is enabled the physical table
name is of the form S:T
+// while the table name is of the form S.T so we need to
query for the
+// PHYSICAL_TABLE link
--- End diff --
Since the linking rows are being re-written I believe
in a follow up JIRA.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
>
- End diff --
Is this running a new scan to find the ancestor links? A comment here might
be good.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.o
and then dropped
+// and clean up any parent->child links or child views
--- End diff --
Remove TODO as isn't this done now?
> Support multi region SYSTEM.CATALOG table
> -
>
> Ke
ld the be covered by one
of the future JIRAs you mentioned?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project:
present
to the physical table. So the
+* viewPhysicalTableRow is null in that case.
+*/
--- End diff --
Fix indentation
> Support multi region SYSTEM.CATALOG ta
Do you think there'd be any issues
if part of the rows for a view were there (i.e. say that the create view
failed, but some of the rows were written)? Might be good to have a test like
this - you could set it up by using HBase APIs to manually delete some rows of
a view.
> Support multi regi
--
Is this ever the case since the index should be in the same schema as it's
table? Or is there a corner case with indexes on views?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
>
iff --
TODO to remove this code in 4.16.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
if (column.equals(SaltingUtil.SALTING_COLUMN) ||
column.isExcluded()) {
--- End diff --
Same comment as before - we should be able to match based on the column
being the first column. Note that the salt column would only be in the base
physical table.
> Support multi
ns.add(pColumn);
+} else if (!pColumn.equals(SaltingUtil.SALTING_COLUMN)) {
--- End diff --
Instead of matching on SALTING_COLUMN, we should stop the loop at 1 above
if table.getSaltBuckets() != null. The code never filters the salt column based
. Move views to their own table
2. Get rid of client side code that is sending the base columns
3. Fix corner case/race condition issues
4. Add code that doesn't write orphaned metadata on major compaction
> Support multi region SYSTEM.CATALOG ta
tadata(schemaName, tableName,
tenantIdBytes);
+ }
--- End diff --
Minor - indentation issue here.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
>
Github user JamesRTaylor commented on the issue:
https://github.com/apache/phoenix/pull/303
+1 to the patch. Great work @twdsilva and @churrodog! I made some minor
comments for some potential follow up work and had a few questions, but let's
get this committed first. I'd recommend
://builds.apache.org/job/PreCommit-PHOENIX-Build/1900/consoleFull
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
>
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva updated PHOENIX-3534:
Attachment: (was: PHOENIX-3534.patch)
> Support multi region SYSTEM.CATALOG ta
Assignee: Thomas D'Silva
>Priority: Major
> Attachments: PHOENIX-3534.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region
> based on the server-side row locks being held for operations that impact a
> table and all of it's v
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva updated PHOENIX-3534:
Attachment: PHOENIX-3534.patch
> Support multi region SYSTEM.CATALOG ta
latest changes from master as well.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
>
t;Assignee: Thomas D'Silva
>Priority: Major
> Attachments: PHOENIX-3534-wip.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region
> based on the server-side row locks being held for operations that impact a
> table a
f normal HBase replication works for System.CHILD_LINK, and all view data
left in System.Catalog starts with tenant_id, then the logic here can be
greatly simplified, similar to how it was before PHOENIX-4229
> Support multi region SYSTEM.CA
cute("ALTER TABLE " + fullTableName + "
DROP COLUMN v1");
+// TODO see if its possibel to prevent the dropping of a column
thats required by a child view (for its view where clause)
+// the view
key of the column
row. We could also do the checkAndPut on the new SYSTEM.CHILD_LINK table
instead of SYSTEM.CATALOG. This would be done from MetadataClient to ensure
that no other client can make a conflicting change before we call addColumn or
dropColumn.
> Support multi regio
than doing a
checkAndPut.
If we do the checkAndPut in MetadataClient to ensure that only one client
is able to add the same column at the same time and then make the rpc to the
server to add the column, do you think think that would scale?
> Support multi region SYSTEM.CATALOG table
&g
e.hashCode());
+ result = prime * result + ((name == null) ? 0 : name.hashCode());
+ return result;
+ }
--- End diff --
Fixed
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
throws SQLException {
+ getDataColumnNames().add(node.getName());
+return null;
+}
+
+ public List getDataColumnNames() {
+ return dataColumnNames;
+ }
+
+}
--- End diff --
Fixed.
> Support multi regi
umnConstantsMatched.length; ++i) {
--- End diff --
I removed this code, its not needed.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/brow
sults.next());
+// TODO enable this after we drop view indexes than need a
dropped column
+//assertNull(results.next());
--- End diff --
This test now passes without commenting the assertNull.
> Support multi region SYSTEM.
update the
+// ORDINAL.POSITIONS to be shifted accordingly.
--- End diff --
I filed PHOENIX-4767 to remove the dedup code. We can stop sending the
parent column metadata in the same release.
> Support multi region SYSTEM.CATA
{
}
}
-
+
+// see PHOENIX-3534, now tables can have duplicate columns and they
are removed implicitly
--- End diff --
This test passes, I have remove the ignore annotation.
> Support multi region SYSTEM.CATALOG ta
esultSet
newResultSet(ResultIterator iterator, RowProjector projector,
-StatementContext context) throws
SQLException {
-return new PhoenixResultSet(new
TenantColumnFilteringIterator(iterator, projector),
-projector, context);
-
erSideTableTimeStamp <
MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP_4_14_0) {
--- End diff --
Done
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/bro
)
{
SEQUENCE_NOT_CASTABLE_TO_AUTO_PARTITION_ID_COLUMN(1086, "44A17",
"Sequence Value not castable to auto-partition id column"),
CANNOT_COERCE_AUTO_PARTITION_ID(1087, "44A18", "Auto-partition id
cannot be coerced"),
+
--- End diff --
Done
>
,
table.getPKColumns().get(0).getTimestamp()));
--- End diff --
We are using the first timestamp of the first pk column, so its guaranteed
to be non null.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key:
/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
> Assignee: Thomas D'Silva
>Priority: Major
> Attachments: PHOENIX-3534-wip.patch
>
>
> Currently Phoenix requires that the SYSTEM.CATALOG tab
I think we should keep the OrphanCleaner code. The idea is that failed
deletions of metadata are transparent. They shouldn't block creation of new
tables. We should also cleanup on compaction.
> Support multi region SYSTEM.CATALOG table
> ---
t was dropped
whose child view metadata wasn't cleaned up and then throw an exception.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-
owKey,
TABLE_FAMILY_BYTES, NULLABLE_BYTES,
+MetaDataProtocol.MIN_TABLE_TIMESTAMP,
+
PInteger.INSTANCE.toBytes(SchemaUtil.getIsNullableInt(column.isNullable();
+// REMARKS
+cells.add(
+KeyValu
in the view or not so its we set the table property value to the one
in the view. This is different from current behavior where if we change a
table property on a base table and a child view hasn't modified the property,
the change is propagated to the child. I filed PHOENIX-4763 to fix this a
(See
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1908/])
PHOENIX-4759 During restart RS that hosts SYSTEM.CATALOG table may get (ssa:
rev e6f9ce0352dbf4f5b1a2d086c1c6068426afc1ac)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/147/])
PHOENIX-4759 During restart RS that hosts SYSTEM.CATALOG table may get (ssa:
rev 24af1040591e088a9b4722ae825762d331d102c2)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write
RS that hosts SYSTEM.CATALOG table may get stuck
> ---
>
> Key: PHOENIX-4759
> URL: https://issues.apache.org/jira/browse/PHOENIX-4759
> Project: Phoenix
> Issue Type: Bug
[
https://issues.apache.org/jira/browse/PHOENIX-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated PHOENIX-4759:
-
Attachment: PHOENIX-4759-2.master.patch
> During restart RS that hosts SYSTEM.CATA
it shortly.
> During restart RS that hosts SYSTEM.CATALOG table may get stuck
> ---
>
> Key: PHOENIX-4759
> URL: https://issues.apache.org/jira/browse/PHOENIX-4759
> Project: Phoeni
these version constants to
MetaDataProtocol since that's where other version info is (and it won't have
outside dependencies). Will you be working up a patch?
> During restart RS that hosts SYSTEM.CATALOG table may get st
that doesn't have
other dependencies. Maybe in MetaDataProtocol or QueryConstants?
{quote}
That sounds like a good idea. No strong feelings from me where they get put :)
> During restart RS that hosts SYSTEM.CATALOG table may get st
with the constants (as it ensures
consistency - these are used in multiple places - and makes the code easier to
read). But we should make sure we put them all in one place that doesn't have
other dependencies. Maybe in MetaDataProtocol or QueryConstants?
> During restart RS that hosts SYSTEM.CATALOG ta
[
https://issues.apache.org/jira/browse/PHOENIX-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Soldatov updated PHOENIX-4759:
-
Attachment: PHOENIX-4759-1.patch
> During restart RS that hosts SYSTEM.CATALOG ta
read is loading PhoenixDatabaseMetaData , we
have a dead lock.
We can break this by removing the dependency between SQLExceptionCode and
PhoenixDatabaseMetaData.
> During restart RS that hosts SYSTEM.CATALOG table may get stuck
> ---
>
>
Sergey Soldatov created PHOENIX-4759:
Summary: During restart RS that hosts SYSTEM.CATALOG table may get
stuck
Key: PHOENIX-4759
URL: https://issues.apache.org/jira/browse/PHOENIX-4759
Project
umnConstantsMatched.length; ++i) {
--- End diff --
should this be removed, or is this still valid?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/
,
table.getPKColumns().get(0).getTimestamp()));
--- End diff --
are we guaranteed to get a non null column back?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.
Exception. If the DROP VIEW is issued without
CASCADE and there are child views the statement fails with a not allowed to
mutate exception. If a drop table without cascade and a create view happens
concurrently we could create the view even though the base table would have
been dropped
.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>
sults.next());
+// TODO enable this after we drop view indexes than need a
dropped column
+//assertNull(results.next());
--- End diff --
what about this TODO?
> Support multi region SYSTEM.
ke there'd be an inherent race condition, but if not, I suppose
keeping the same behavior is fine. What about preventing the drop of a table
with child views?
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PH
to the client when calling
dropColumn, so that we drop the local or view index data I think. We don't need
to pass sharedTablesToDelete back to the client when dropping a table.
> Support multi region SYSTEM.CATALOG table
> -
>
>
, but we passed this back
because I believe we don't know on the client all of the physical index tables
to delete. I think we have a test for this.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
>
ableType != PTableType.VIEW) {
+parentLockKey = SchemaUtil.getTableKey(tenantIdBytes,
schemaName, parentTableName);
--- End diff --
We only lock the parent for indexes.
> Support multi region SYSTEM.CATALOG table
> -
>
>
ableType != PTableType.VIEW) {
+parentLockKey = SchemaUtil.getTableKey(tenantIdBytes,
schemaName, parentTableName);
--- End diff --
Shouldn't need this parentLockKey any longer, yes? Or is this only for
indexes?
> Support multi region SYSTEM.CA
e info in the same
release that SYSTEM.CATALOG becomes splittable? Seems like yes.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
used to prevent a base table column from being dropped if the
column was used in a child view index.
It's possible to detect this and prevent the column drop even with
splittable system.catalog if you think we should maintain the existing behavior.
> Support multi region SYSTEM.CATALOG tab
1 - 100 of 159 matches
Mail list logo
