[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu closed the pull request at: https://github.com/apache/phoenix/pull/3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on the pull request: https://github.com/apache/phoenix/pull/3#issuecomment-48887938 @ramkrish86 - would you have time to check this in to master? We can add this minor check in a follow up commit. @chrajeshbabu - will an alternate patch be required for the 4.0 branch? 4.0 should match master, but it seems to be ever so slightly different. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/3#discussion_r14873050
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIterators.java ---
@@ -238,7 +240,12 @@ private void
doColumnProjectionOptimization(StatementContext context, Scan scan,
minMaxRange =
SaltingUtil.addSaltByte(split.getLowerRange(), minMaxRange);
split = split.intersect(minMaxRange);
}
-}
+} else if (localIndex) {
--- End diff --
In a follow up commit, we should look to commonize the above salting logic
with this logic for local indexes. It's doing the same thing, it's just adding
one byte for the salt byte while the local indexing is adding the region start
key.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on the pull request: https://github.com/apache/phoenix/pull/3#issuecomment-48941536 @ramkrish86 - just add a .patch to the url for the pull request and you'll have the patch file to apply against the normal/updatable repo. The github repo is read-only for everyone. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on the pull request: https://github.com/apache/phoenix/pull/3#issuecomment-48945719 bq. Just add the check to disable creating local indexes on a table with immutable rows and then let's check this in. Changed pull request to disallow local index on immutable rows and added some test cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request: https://github.com/apache/phoenix/pull/3#discussion_r14897879 --- Diff: phoenix-core/src/main/java/org/apache/phoenix/expression/ExpressionType.java --- @@ -169,6 +170,7 @@ SQLViewTypeFunction(SQLViewTypeFunction.class), ExternalSqlTypeIdFunction(ExternalSqlTypeIdFunction.class), ConvertTimezoneFunction(ConvertTimezoneFunction.class), +SQLIndexTypeFunction(SQLIndexTypeFunction.class), --- End diff -- done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/3#discussion_r14898287
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java
---
@@ -140,7 +142,14 @@ public boolean apply(HRegionLocation location) {
// distributed across regions, using this scheme compensates for
regions that
// have more rows than others, by applying tighter splits and
therefore spawning
// off more scans over the overloaded regions.
-int splitsPerRegion = regions.size() = targetConcurrency ? 1 :
(regions.size() targetConcurrency / 2 ? maxConcurrency : targetConcurrency) /
regions.size();
+PTable table = tableRef.getTable();
--- End diff --
bq. why do we need this to be different for local indexes? Seems like we
could run parallel scans over part of each region just like we do with other
scans, no?
Index table rows have region start key as prefix so even if we split region
key ranges into multiple and scan parallelly,only first scanner gives the
results and remaining all the scanners just return nothing. So number of splits
of local index region setting to 1.
if we want to start multiple scanners for local index into multiple parts
then we need to split the scan ranges into multiple.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/3#discussion_r14898383
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java
---
@@ -140,7 +142,14 @@ public boolean apply(HRegionLocation location) {
// distributed across regions, using this scheme compensates for
regions that
// have more rows than others, by applying tighter splits and
therefore spawning
// off more scans over the overloaded regions.
-int splitsPerRegion = regions.size() = targetConcurrency ? 1 :
(regions.size() targetConcurrency / 2 ? maxConcurrency : targetConcurrency) /
regions.size();
+PTable table = tableRef.getTable();
--- End diff --
bq. getSplitsPerRegion method to ParallelIteratorRegionSplitter so we can
move this special case to your new implementation for local indexes.
Current patch added getSplitsPerRegion method to
ParallelIteratorRegionSplitter and returning one in
LocalIndexParallelIteratorRegionSplitter and removed the changes in other
splitter.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on the pull request: https://github.com/apache/phoenix/pull/3#issuecomment-48949296 Resolved the conflicts after PHOENIX-1002 also. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14858124
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java
---
@@ -54,7 +55,8 @@ protected
SkipRangeParallelIteratorRegionSplitter(StatementContext context, Tabl
public ListHRegionLocation filterRegions(ListHRegionLocation
allTableRegions, final ScanRanges ranges) {
IterableHRegionLocation regions;
-if (ranges == ScanRanges.EVERYTHING) {
+if (ranges == ScanRanges.EVERYTHING
--- End diff --
Even with the change skip scan will be used James. The change is required
because the key ranges generated by compiler won't be in the local index
regions key range because local index rows have prefixed region start key
extra. Without the change mostly no region will be selected for scanning.
QueryIT#testSimpleInListStatement is the test case verifies the same.
Here is the explain query result.
CLIENT PARALLEL 4-WAY SKIP SCAN ON 2 KEYS OVER _LOCAL_IDX_ATABLE [-32768,2]
- [-32768,4]
SERVER FILTER BY FIRST KEY ONLY AND ORGANIZATION_ID = '00D3XHP'
CLIENT MERGE SORT
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on the pull request: https://github.com/apache/phoenix/pull/1#issuecomment-48839233 bq. Cleanest might be to just implement a simple ParallelIteratorRegionSplitter for use when a local index is used that just returns all regions: I will add new ParallelIteratorRegionSplitter for local index and remove the unnecessary changes in SkipRangeParallelIteratorRegionSplitter/DefaultParallelIteratorRegionSplitter. Then I will submit another pull request. Thanks @JamesRTaylor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
GitHub user chrajeshbabu opened a pull request: https://github.com/apache/phoenix/pull/3 PHOENIX-933 Local index support to Phoenix Updated pull request after resolving conflicts and handling James review comments. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chrajeshbabu/phoenix master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/3.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3 commit 014ec218b151ef960347b875b373828ad8fe8681 Author: James Taylor jamestay...@apache.org Date: 2014-04-17T01:28:18Z Add local index to grammar and metadata commit 1855a8d1c6b2064d81f5fffc27c18342599cf7b0 Author: James Taylor jamestay...@apache.org Date: 2014-04-18T23:31:29Z PHOENIX-936 Custom load balancer to colocate user table regions and index table regions (Rajeshbabu) commit 1bc8b57801e299635ac601aea5dc7ba8d4915a2f Author: James Taylor jamestay...@apache.org Date: 2014-04-26T04:36:03Z PHOENIX-935 create local index table with the same split keys of user table (Rajeshbabu) commit b02df4f054f78a866f25d9beb4e7fc2242df2a1f Author: James Taylor jamestay...@apache.org Date: 2014-04-28T05:26:27Z PHOENIX-935 create local index table with the same split keys of user table (Rajeshbabu) commit 8f3f8100210538d0c162cbbc5e02efc63db52d31 Author: James Taylor jamestay...@apache.org Date: 2014-04-28T16:31:31Z PHOENIX-955 Skip region start key at beginning of local index rows (JamesTaylor) commit 8e35bc99068da33db4e2019d9bc65d8d78bee4d2 Author: James Taylor jtay...@salesforce.com Date: 2014-05-08T06:23:37Z PHOENIX-937 Handle puts on local index table (Rajeshbabu) commit cf78def0579f51a5342bcfbff802902478052d25 Author: James Taylor jtay...@salesforce.com Date: 2014-05-23T03:48:01Z PHOENIX-994 Handle scans on local index table in case any best fit covering local index available (Rajeshbabu) commit 835bcf675db9435da1335bbfb6d55d6d7edd86b3 Author: James Taylor jtay...@salesforce.com Date: 2014-05-27T19:51:56Z PHOENIX-1004 'drop index' should delete index data from local index table (Rajeshbabu) commit 2edd2b6a19deef0847086da2c7b30250ed2907c7 Author: James Taylor jtay...@salesforce.com Date: 2014-06-09T18:11:35Z PHOENIX-1038 Dynamically add INDEX_TYPE column to SYSTEM.CATALOG if not already there commit 0408b333780f4ce428e5d9b018e5cd13934d0f84 Author: Ramkrishna ramkrishna.s.vasude...@intel.com Date: 2014-07-04T10:19:41Z PHOENIX-1015 Support joining back to data table row from local index when query condition involves leading columns in local index (Rajeshbabu) commit 3df832e73e138320a30e5c5d0c80680b79de6358 Author: Rajeshbabu Chintaguntla rajeshbabu.chintagun...@huawei.com Date: 2014-07-12T17:14:44Z PHOENIX-1015 Support joining back to data table row from local index when query condition involves leading columns in local index commit 84097757a3173adc2a3b464901b79f35bc9d07cb Author: Rajeshbabu Chintaguntla rajeshbabu.chintagun...@huawei.com Date: 2014-07-13T17:26:13Z PHOENIX-1015 Support joining back to data table row from local index when query condition involves leading columns in local index --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/3#discussion_r14860092
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/compile/IndexStatementRewriter.java
---
@@ -96,6 +96,11 @@ public ParseNode visit(ColumnParseNode node) throws
SQLException {
String indexColName = IndexUtil.getIndexColumnName(dataCol);
// Same alias as before, but use the index column name instead of
the data column name
+// ColumnParseNode cannot be resolved. When this occurs, add the
dataColRef to a list
--- End diff --
Remove this comment block as it's not relevant any longer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/3#discussion_r14860610
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java
---
@@ -140,7 +142,14 @@ public boolean apply(HRegionLocation location) {
// distributed across regions, using this scheme compensates for
regions that
// have more rows than others, by applying tighter splits and
therefore spawning
// off more scans over the overloaded regions.
-int splitsPerRegion = regions.size() = targetConcurrency ? 1 :
(regions.size() targetConcurrency / 2 ? maxConcurrency : targetConcurrency) /
regions.size();
+PTable table = tableRef.getTable();
--- End diff --
Please add TODO to add getSplitsPerRegion method to
ParallelIteratorRegionSplitter so we can move this special case to your new
implementation for local indexes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14852114
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/BaseTenantSpecificViewIndexIT.java
---
@@ -117,26 +139,38 @@ private Connection createTenantConnection(String
tenantId) throws SQLException {
private void verifyViewData(Connection conn, String valuePrefix)
throws SQLException {
String query = SELECT k1, k2, v2 FROM v WHERE v2=' + valuePrefix
+ v2-1';
ResultSet rs = conn.createStatement().executeQuery(query);
-assertTrue(rs.next());
-assertEquals(1, rs.getInt(1));
-assertEquals(1, rs.getInt(2));
-assertEquals(valuePrefix + v2-1, rs.getString(3));
-assertTrue(rs.next());
-assertEquals(1, rs.getInt(1));
-assertEquals(3, rs.getInt(2));
-assertEquals(valuePrefix + v2-1, rs.getString(3));
-assertTrue(rs.next());
-assertEquals(1, rs.getInt(1));
-assertEquals(5, rs.getInt(2));
-assertEquals(valuePrefix + v2-1, rs.getString(3));
-assertTrue(rs.next());
-assertEquals(1, rs.getInt(1));
-assertEquals(7, rs.getInt(2));
-assertEquals(valuePrefix + v2-1, rs.getString(3));
-assertTrue(rs.next());
-assertEquals(1, rs.getInt(1));
-assertEquals(9, rs.getInt(2));
-assertEquals(valuePrefix + v2-1, rs.getString(3));
+ListListObject expectedResultsA = Lists.newArrayList(
+Arrays.ObjectasList(1,1, valuePrefix + v2-1),
+Arrays.ObjectasList(1,3, valuePrefix + v2-1),
+Arrays.ObjectasList(1,5, valuePrefix + v2-1),
+Arrays.ObjectasList(1,7, valuePrefix + v2-1),
+Arrays.ObjectasList(1,9, valuePrefix + v2-1));
+assertValuesEqualsResultSet(rs,expectedResultsA);
assertFalse(rs.next());
}
+
--- End diff --
This code looks familiar. If it matches original (I think from QueryIT),
can you move it into the base test class instead of copy/paste?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14852129
--- Diff:
phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexIT.java
---
@@ -51,21 +51,38 @@ public static void doSetup() throws Exception {
props.put(QueryServices.MAX_INTRA_REGION_PARALLELIZATION_ATTRIB,
Integer.toString(1));
// Forces server cache to be used
props.put(QueryServices.INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB,
Integer.toString(2));
+props.put(QueryServices.DROP_METADATA_ATTRIB,
Boolean.toString(true));
// Must update config before starting server
startServer(getUrl(), new
ReadOnlyProps(props.entrySet().iterator()));
}
-
+
@Test
public void testIndexWithNullableFixedWithCols() throws Exception {
+testIndexWithNullableFixedWithCols(false);
+}
+
+@Test
+public void testLocalIndexWithNullableFixedWithCols() throws Exception
{
+testIndexWithNullableFixedWithCols(true);
+}
+
+private void testIndexWithNullableFixedWithCols(boolean localIndex)
throws Exception {
Properties props = new Properties(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
conn.setAutoCommit(false);
try {
createTestTable();
populateTestTable();
-String ddl = CREATE INDEX + INDEX_TABLE_NAME + ON +
DATA_TABLE_FULL_NAME
-+ (char_col1 ASC, int_col1 ASC)
-+ INCLUDE (long_col1, long_col2);
+String ddl = null;
+if(localIndex){
+ddl = CREATE INDEX + INDEX_TABLE_NAME + ON +
DATA_TABLE_FULL_NAME
--- End diff --
Looks like the if and the else branch match. If that's expected/correct,
can you get rid of the if statement?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14853783
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanRegionObserver.java
---
@@ -100,7 +109,7 @@ public static void serializeIntoScan(Scan scan, int
thresholdBytes, int limit, L
}
}
-public static OrderedResultIterator deserializeFromScan(Scan scan,
RegionScanner s) {
+public static OrderedResultIterator deserializeFromScan(Scan scan,
RegionScanner s, int offset) {
--- End diff --
bq. There's a bit more you need to do to handle ORDER BY correctly. It'd be
for the case in which a data column was referenced in the ORDER BY while the
index table is being used to satisfy the query.
This is working fine James. I have added test case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14854253
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanRegionObserver.java
---
@@ -100,7 +109,7 @@ public static void serializeIntoScan(Scan scan, int
thresholdBytes, int limit, L
}
}
-public static OrderedResultIterator deserializeFromScan(Scan scan,
RegionScanner s) {
+public static OrderedResultIterator deserializeFromScan(Scan scan,
RegionScanner s, int offset) {
--- End diff --
Excellent. Would you mind updating this pull request? I think it's ready to
get committed, no?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14834867
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/compile/CreateIndexCompiler.java
---
@@ -47,6 +51,21 @@ public MutationPlan compile(final CreateIndexStatement
create) throws SQLExcepti
final StatementContext context = new StatementContext(statement,
resolver, scan);
ExpressionCompiler expressionCompiler = new
ExpressionCompiler(context);
ListParseNode splitNodes = create.getSplitNodes();
+if (create.getIndexType() == IndexType.LOCAL) {
+if (!splitNodes.isEmpty()) {
+throw new
SQLExceptionInfo.Builder(SQLExceptionCode.CANNOT_SPLIT_LOCAL_INDEX)
+.build().buildException();
+}
+if (create.getProps() != null create.getProps().get() !=
null) {
+ListPairString, Object list =
create.getProps().get();
--- End diff --
corrected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14834896
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionCompiler.java
---
@@ -282,7 +286,7 @@ public Expression visitLeave(FunctionParseNode node,
ListExpression children)
children = node.validate(children, context);
Expression expression = node.create(children, context);
ImmutableBytesWritable ptr = context.getTempPtr();
-if (node.isStateless()) {
+if (node.isStateless() expression.isDeterministic()) {
--- End diff --
Yes James. This change already there in master branch
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14834954
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/compile/IndexStatementRewriter.java
---
@@ -96,6 +96,12 @@ public ParseNode visit(ColumnParseNode node) throws
SQLException {
String indexColName = IndexUtil.getIndexColumnName(dataCol);
// Same alias as before, but use the index column name instead of
the data column name
+// TODO: add dataColRef as an alternate ColumnParseNode in the
case that the index
--- End diff --
removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user chrajeshbabu commented on a diff in the pull request: https://github.com/apache/phoenix/pull/1#discussion_r14850777 --- Diff: phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java --- @@ -366,6 +384,21 @@ private RegionScanner scanUnordered(ObserverContextRegionCoprocessorEnvironment env, ScanUtil.getTenantId(scan), aggregators, estDistVals); +byte[] localIndexBytes = scan.getAttribute(LOCAL_INDEX_BUILD); --- End diff -- Moved the changes outside of scanOrdered/scanUnordered and passing through necessary info through calls. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] phoenix pull request: PHOENIX-933 Local index support to Phoenix
Github user JamesRTaylor commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/1#discussion_r14762643
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/compile/TrackOrderPreservingExpressionCompiler.java
---
@@ -69,6 +70,7 @@
boolean isSharedViewIndex = table.getViewIndexId() != null;
// TODO: util for this offset, as it's computed in numerous places
positionOffset = (isSalted ? 1 : 0) + (isMultiTenant ? 1 : 0) +
(isSharedViewIndex ? 1 : 0);
+this.isOrderPreserving = table.getIndexType() != IndexType.LOCAL;
--- End diff --
One thing that's necessary, though, to maintain rows in row key order is to
modify ScanPlan.java:118 to do a merge sort instead of a concat:
if ((isSalted || isLocalIndex)
(context.getConnection().getQueryServices().getProps().getBoolean(
QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB,
QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE) ||
orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY ||
orderBy == OrderBy.REV_ROW_KEY_ORDER_BY)) { // ORDER
BY was optimized out b/c query is in row key order
scanner = new MergeSortRowKeyResultIterator(iterators,
SaltingUtil.NUM_SALTING_BYTES, orderBy == OrderBy.REV_ROW_KEY_ORDER_BY);
} else {
scanner = new ConcatResultIterator(iterators);
}
Local indexes are similar to salted tables in that the parallel scans will
all be within a region, ordered correctly. As long as we do a merge sort across
the results of these scans, the rows will be ordered correctly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
