[jira] [Commented] (PHOENIX-4530) Do not collect delete markers during major compaction of table with disabled mutable indexes

2018-02-20 Thread Vincent Poon (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370862#comment-16370862
 ] 

Vincent Poon commented on PHOENIX-4530:
---

[~jamestaylor] v2 patch attached:
 * I moved the logic back to UngroupedAggregateRegionObserver to make it so it 
works for compaction of either the data table or index tables
 * PartialIndexRebuilderIT#testCompactionDuringRebuild() is no longer valid, 
since it tests for disabling of the index and we no longer do that, so I 
removed it
 * HBase uses the HStore directly to do its compaction testing - I even call a 
method called compactRecentForTesting().  I did see Phoenix's 
TestUtil.doMajorCompaction(), but it's pretty crude - it does an async request 
through the HBaseAdmin, then polls the table to check the status.  Furthermore, 
the logic in there doesn't work for our use case, because it does a Put then a 
Delete, then compacts and polls to see if the cells were removed.  In our case, 
we're not deleting the cells, so the test will never pass and you get an 
infinite loop.
 * Added a TestUtil.getRawRowCount()

> Do not collect delete markers during major compaction of table with disabled 
> mutable indexes
> 
>
> Key: PHOENIX-4530
> URL: https://issues.apache.org/jira/browse/PHOENIX-4530
> Project: Phoenix
>  Issue Type: Improvement
> Environment:  
>Reporter: James Taylor
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4530.master.v1.patch, 
> PHOENIX-4530.master.v2.patch
>
>
> If major compaction occurs on a table with mutable indexes that have the 
> INDEX_DISABLE_TIMESTAMP set, we currently permanently disable the index, 
> forcing it to be manually rebuilt from scratch. This is to prevent it from 
> potentially being corrupted as we need the delete markers to remain in order 
> to guarantee the data table and index table remain in sync.
> An alternate approach (mentioned by [~an...@apache.org] during review) is to 
> detect this case in a pre-compaction hook and set the compaction up so that 
> delete markers are not removed. This would have the advantage that we 
> wouldn't have to permanently disable the index and rebuild it from scratch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4530) Do not collect delete markers during major compaction of table with disabled mutable indexes

2018-02-20 Thread Vincent Poon (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vincent Poon updated PHOENIX-4530:
--
Attachment: PHOENIX-4530.master.v2.patch

> Do not collect delete markers during major compaction of table with disabled 
> mutable indexes
> 
>
> Key: PHOENIX-4530
> URL: https://issues.apache.org/jira/browse/PHOENIX-4530
> Project: Phoenix
>  Issue Type: Improvement
> Environment:  
>Reporter: James Taylor
>Assignee: Vincent Poon
>Priority: Major
> Attachments: PHOENIX-4530.master.v1.patch, 
> PHOENIX-4530.master.v2.patch
>
>
> If major compaction occurs on a table with mutable indexes that have the 
> INDEX_DISABLE_TIMESTAMP set, we currently permanently disable the index, 
> forcing it to be manually rebuilt from scratch. This is to prevent it from 
> potentially being corrupted as we need the delete markers to remain in order 
> to guarantee the data table and index table remain in sync.
> An alternate approach (mentioned by [~an...@apache.org] during review) is to 
> detect this case in a pre-compaction hook and set the compaction up so that 
> delete markers are not removed. This would have the advantage that we 
> wouldn't have to permanently disable the index and rebuild it from scratch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4231) Support restriction of remote UDF load sources

2018-02-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370810#comment-16370810
 ] 

ASF GitHub Bot commented on PHOENIX-4231:
-

Github user aertoria commented on a diff in the pull request:

https://github.com/apache/phoenix/pull/292#discussion_r169509766
  
--- Diff: 
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java ---
@@ -907,10 +909,15 @@ public MutationState execute() throws SQLException {
 try {
 FileSystem fs = 
dynamicJarsDirPath.getFileSystem(conf);
 List jarPaths = getJarPaths();
-for (LiteralParseNode jarPath : jarPaths) {
-File f = new File((String) jarPath.getValue());
-fs.copyFromLocalFile(new 
Path(f.getAbsolutePath()), new Path(
-dynamicJarsDir + f.getName()));
+for (LiteralParseNode jarPathNode : jarPaths) {
+  String jarPathName = (String) 
jarPathNode.getValue();
+  File f = new File(jarPathName);
+  Path dynamicJarsDirPathWithJar = new 
Path(dynamicJarsDir + f.getName());
+  // Copy the jar (can be local or on HDFS) to the 
hbase.dynamic.jars.dir directory.
+  // Note that this does not support HDFS URIs 
without scheme and authority.
+  Path jarPath = new Path(jarPathName);
+  FileUtil.copy(jarPath.getFileSystem(conf), 
jarPath, fs, dynamicJarsDirPathWithJar,
--- End diff --

This copy happen at phoenixStatment.java which is at client side. Shouldn't 
the checking be happening at Server side


> Support restriction of remote UDF load sources 
> ---
>
> Key: PHOENIX-4231
> URL: https://issues.apache.org/jira/browse/PHOENIX-4231
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Chinmay Kulkarni
>Priority: Major
>
> When allowUserDefinedFunctions is true, users can load UDFs remotely via a 
> jar file from any HDFS filesystem reachable on the network. The setting 
> hbase.dynamic.jars.dir can be used to restrict locations for jar loading but 
> is only applied to jars loaded from the local filesystem.  We should 
> implement support for similar restriction via configuration for jars loaded 
> via hdfs:// URIs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix pull request #292: PHOENIX-4231: Support restriction of remote UDF l...

2018-02-20 Thread aertoria 
Github user aertoria commented on a diff in the pull request:

https://github.com/apache/phoenix/pull/292#discussion_r169509766
  
--- Diff: 
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java ---
@@ -907,10 +909,15 @@ public MutationState execute() throws SQLException {
 try {
 FileSystem fs = 
dynamicJarsDirPath.getFileSystem(conf);
 List jarPaths = getJarPaths();
-for (LiteralParseNode jarPath : jarPaths) {
-File f = new File((String) jarPath.getValue());
-fs.copyFromLocalFile(new 
Path(f.getAbsolutePath()), new Path(
-dynamicJarsDir + f.getName()));
+for (LiteralParseNode jarPathNode : jarPaths) {
+  String jarPathName = (String) 
jarPathNode.getValue();
+  File f = new File(jarPathName);
+  Path dynamicJarsDirPathWithJar = new 
Path(dynamicJarsDir + f.getName());
+  // Copy the jar (can be local or on HDFS) to the 
hbase.dynamic.jars.dir directory.
+  // Note that this does not support HDFS URIs 
without scheme and authority.
+  Path jarPath = new Path(jarPathName);
+  FileUtil.copy(jarPath.getFileSystem(conf), 
jarPath, fs, dynamicJarsDirPathWithJar,
--- End diff --

This copy happen at phoenixStatment.java which is at client side. Shouldn't 
the checking be happening at Server side


---

[jira] [Commented] (PHOENIX-4414) Exception while using database metadata commands on tenant specific connection

2018-02-20 Thread Mujtaba Chohan (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370789#comment-16370789
 ] 

Mujtaba Chohan commented on PHOENIX-4414:
-

[~rajeshbabu] done.

> Exception while using database metadata commands on tenant specific connection
> --
>
> Key: PHOENIX-4414
> URL: https://issues.apache.org/jira/browse/PHOENIX-4414
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Mujtaba Chohan
>Assignee: Mujtaba Chohan
>Priority: Minor
> Attachments: PHOENIX-4414.patch, PHOENIX-4414_v2.patch
>
>
> This is when using tenant specific connection from Sqlline.
> {noformat}
> Error: ERROR 602 (42P00): Syntax error. Missing "LPAREN" at line 2, column 
> 746. (state=42P00,code=602)
> org.apache.phoenix.exception.PhoenixParserException: ERROR 602 (42P00): 
> Syntax error. Missing "LPAREN" at line 2, column 746.
>   at 
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
>   at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1529)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.parseStatement(PhoenixStatement.java:1612)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeQuery(PhoenixStatement.java:1653)
>   at 
> org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.getColumns(PhoenixDatabaseMetaData.java:557)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view

2018-02-20 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370707#comment-16370707
 ] 

James Taylor commented on PHOENIX-4333:
---

wip4 patch fixes some corner cases and cleans up where we update stats. It's 
complicated by considering the current guidepost which is the end of a scan 
range, since if the gp is equal to the end region key, we should not be 
including it. Next patch will add a guidepost when it's used as the start of 
the scan which should be simpler.

> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
>  Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch, 
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch, 
> PHOENIX-4333_wip3.patch, PHOENIX-4333_wip4.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate 
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view 
> yield partial results (only contains stats for B,1) which are incorrect even 
> though it shows updated timestamp as current.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view

2018-02-20 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4333:
--
Attachment: (was: PHOENIX-4333_wip4.patch)

> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
>  Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch, 
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch, 
> PHOENIX-4333_wip3.patch, PHOENIX-4333_wip4.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate 
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view 
> yield partial results (only contains stats for B,1) which are incorrect even 
> though it shows updated timestamp as current.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view

2018-02-20 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4333:
--
Attachment: PHOENIX-4333_wip4.patch

> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
>  Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch, 
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch, 
> PHOENIX-4333_wip3.patch, PHOENIX-4333_wip4.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate 
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view 
> yield partial results (only contains stats for B,1) which are incorrect even 
> though it shows updated timestamp as current.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4607) Allow PhoenixInputFormat to use tenant-specific connections

2018-02-20 Thread Geoffrey Jacoby (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Geoffrey Jacoby updated PHOENIX-4607:
-
Attachment: PHOENIX-4607.patch

> Allow PhoenixInputFormat to use tenant-specific connections
> ---
>
> Key: PHOENIX-4607
> URL: https://issues.apache.org/jira/browse/PHOENIX-4607
> Project: Phoenix
>  Issue Type: New Feature
>Affects Versions: 4.13.0
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Major
> Attachments: PHOENIX-4607.patch
>
>
> When using Phoenix's MapReduce integration, the actual connections for the 
> SELECT query are created by PhoenixInputFormat. While PhoenixInputFormat has 
> support for a few connection properties such as SCN, a TenantId is not one of 
> them. 
> Add the ability to specify a TenantId for the PhoenixInputFormat's 
> connections to use. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-3837) Unable to set property on an index with Alter statement

2018-02-20 Thread Ethan Wang (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370681#comment-16370681
 ] 

Ethan Wang commented on PHOENIX-3837:
-

[~rajeshbabu]

is 5.x-HBase-2.0 branch broken right now? I did a fresh pull and mvn build 
failed at

 
/Users/ethan.wang/Desktop/FRESH/phoenix/phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java:[254,93]
 cannot find symbol

[ERROR]   symbol:   variable indexMutations

[ERROR]   location: class org.apache.phoenix.compile.DeleteCompiler

 

Also, I saw PHOENIX-2566 has been back ported, but not completely. I will try 
to back port the rest of that patch.

"PHOENIX-2566 Support NOT NULL constraint for any column for immutable table 
(addendum 2)"

 

> Unable to set property on an index with Alter statement
> ---
>
> Key: PHOENIX-3837
> URL: https://issues.apache.org/jira/browse/PHOENIX-3837
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3837-Merged_With_5.x-2.0-and-Master.patch, 
> PHOENIX-3837.patch
>
>
> {{ALTER INDEX IDX_T ON T SET GUIDE_POSTS_WIDTH=1}}
> {noformat}
> Error: ERROR 601 (42P00): Syntax error. Encountered "SET" at line 1, column 
> 102. (state=42P00,code=601)
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): 
> Syntax error. Encountered "SET" at line 1, column 102.
> at 
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at 
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1299)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view

2018-02-20 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4333:
--
Attachment: PHOENIX-4333_wip4.patch

> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
>  Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch, 
> PHOENIX-4333_v2.patch, PHOENIX-4333_wip1.patch, PHOENIX-4333_wip2.patch, 
> PHOENIX-4333_wip3.patch, PHOENIX-4333_wip4.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate 
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view 
> yield partial results (only contains stats for B,1) which are incorrect even 
> though it shows updated timestamp as current.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (PHOENIX-4601) Perform server-side retries if client version < 4.14

2018-02-20 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reassigned PHOENIX-4601:
-

Assignee: Vincent Poon

> Perform server-side retries if client version < 4.14
> 
>
> Key: PHOENIX-4601
> URL: https://issues.apache.org/jira/browse/PHOENIX-4601
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Vincent Poon
>Priority: Major
>
> The client version is now available on the server side when index maintenance 
> is being performed. Given that this information is available, we should 
> conditionally retry on the server depending on the client version (instead of 
> relying on the operator to manually update the config after clients have been 
> upgraded). 
> With  PHOENIX-4613, the client version has been threaded through to the 
> IndexCommitter.write() method. All that's left to do is:
> - Always set the config on the server side to have no HBase retries.
> - Add catch of IOException and conditionally call the retrying exception 
> handler code based on clientVersion < 4.14.0 in 
> TrackingParallelWriterIndexCommitter and ParallelWriterIndexCommitter.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index

2018-02-20 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4531:
--
Fix Version/s: 5.0.0

> Delete on a table with a global mutable index can issue client-side deletes 
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:  
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4531.v1.master.patch, 
> PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, 
> PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, 
> PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in 
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table" 
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the 
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-3837) Unable to set property on an index with Alter statement

2018-02-20 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370539#comment-16370539
 ] 

James Taylor commented on PHOENIX-3837:
---

Yes, [~aertoria] - the 5.x-HBase-2.0 branch.

> Unable to set property on an index with Alter statement
> ---
>
> Key: PHOENIX-3837
> URL: https://issues.apache.org/jira/browse/PHOENIX-3837
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3837-Merged_With_5.x-2.0-and-Master.patch, 
> PHOENIX-3837.patch
>
>
> {{ALTER INDEX IDX_T ON T SET GUIDE_POSTS_WIDTH=1}}
> {noformat}
> Error: ERROR 601 (42P00): Syntax error. Encountered "SET" at line 1, column 
> 102. (state=42P00,code=601)
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): 
> Syntax error. Encountered "SET" at line 1, column 102.
> at 
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at 
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1299)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-3837) Unable to set property on an index with Alter statement

2018-02-20 Thread Ethan Wang (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370533#comment-16370533
 ] 

Ethan Wang commented on PHOENIX-3837:
-

[~rajeshbabu]

is 5.x-HBase-2.0 the branch you will need to catch up? I'm try to make that in 
today 

> Unable to set property on an index with Alter statement
> ---
>
> Key: PHOENIX-3837
> URL: https://issues.apache.org/jira/browse/PHOENIX-3837
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.10.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-3837-Merged_With_5.x-2.0-and-Master.patch, 
> PHOENIX-3837.patch
>
>
> {{ALTER INDEX IDX_T ON T SET GUIDE_POSTS_WIDTH=1}}
> {noformat}
> Error: ERROR 601 (42P00): Syntax error. Encountered "SET" at line 1, column 
> 102. (state=42P00,code=601)
> org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): 
> Syntax error. Encountered "SET" at line 1, column 102.
> at 
> org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33)
> at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:111)
> at 
> org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1299)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Save the date: ApacheCon North America, September 24-27 in Montréal

2018-02-20 Thread Rich Bowen 

Dear Apache Enthusiast,

(You’re receiving this message because you’re subscribed to a user@ or 
dev@ list of one or more Apache Software Foundation projects.)


We’re pleased to announce the upcoming ApacheCon [1] in Montréal, 
September 24-27. This event is all about you — the Apache project community.


We’ll have four tracks of technical content this time, as well as lots 
of opportunities to connect with your project community, hack on the 
code, and learn about other related (and unrelated!) projects across the 
foundation.


The Call For Papers (CFP) [2] and registration are now open. Register 
early to take advantage of the early bird prices and secure your place 
at the event hotel.


Important dates
March 30: CFP closes
April 20: CFP notifications sent
	August 24: Hotel room block closes (please do not wait until the last 
minute)


Follow @ApacheCon on Twitter to be the first to hear announcements about 
keynotes, the schedule, evening events, and everything you can expect to 
see at the event.


See you in Montréal!

Sincerely, Rich Bowen, V.P. Events,
on behalf of the entire ApacheCon team

[1] http://www.apachecon.com/acna18
[2] https://cfp.apachecon.com/conference.html?apachecon-north-america-2018

[jira] [Created] (PHOENIX-4620) Fix compilation issues with 2.0.0-beta-2-SNAPSHOT

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 
Rajeshbabu Chintaguntla created PHOENIX-4620:


 Summary: Fix compilation issues with 2.0.0-beta-2-SNAPSHOT
 Key: PHOENIX-4620
 URL: https://issues.apache.org/jira/browse/PHOENIX-4620
 Project: Phoenix
  Issue Type: Sub-task
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
 Fix For: 5.0.0






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369975#comment-16369975
 ] 

Rajeshbabu Chintaguntla commented on PHOENIX-4531:
--

Committed to 5.x branch. [~jamestaylor] can we resolve this?

> Delete on a table with a global mutable index can issue client-side deletes 
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:  
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4531.v1.master.patch, 
> PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, 
> PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, 
> PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in 
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table" 
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the 
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369968#comment-16369968
 ] 

Rajeshbabu Chintaguntla commented on PHOENIX-4531:
--

[~jamestaylor] This requires PHOENIX-4386 in 5.x branch which I have committed 
and uploaded patch on top of 5.x branch. Going to commit it.

> Delete on a table with a global mutable index can issue client-side deletes 
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:  
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4531.v1.master.patch, 
> PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, 
> PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, 
> PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in 
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table" 
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the 
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4531) Delete on a table with a global mutable index can issue client-side deletes against the index

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajeshbabu Chintaguntla updated PHOENIX-4531:
-
Attachment: PHOENIX-4531_5.x-HBase-2.0.patch

> Delete on a table with a global mutable index can issue client-side deletes 
> against the index
> -
>
> Key: PHOENIX-4531
> URL: https://issues.apache.org/jira/browse/PHOENIX-4531
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.13.0
> Environment:  
>Reporter: Vincent Poon
>Assignee: Vincent Poon
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4531.v1.master.patch, 
> PHOENIX-4531.v3.master.patch, PHOENIX-4531.v4.master.patch, 
> PHOENIX-4531.v5.master.patch, PHOENIX-4531_5.x-HBase-2.0.patch, 
> PHOENIX-4531_v1.patch, PHOENIX-4531_v2.patch, PartialIndexRebuilderIT.java
>
>
> For a table with a global mutable index, I found the following result in 
> client-side deletes against both the data table and index table.
> "DELETE FROM data_table" 
> "DELETE FROM data_table WHERE indexed_col='v'"
> We only need the delete to be issued against the data table, because
> 1) It's redundant since a delete against the index will be issued on the 
> server side when we process the delete of the data table row
> 2) Deletes issued from the client-side won't have the index failure policy



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4386) Calculate the estimatedSize of MutationState using Map<TableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369953#comment-16369953
 ] 

Rajeshbabu Chintaguntla commented on PHOENIX-4386:
--

This is missing in 5.x branch. Here is the patch on top of it. 

> Calculate the estimatedSize of MutationState using Map Map> mutations
> ---
>
> Key: PHOENIX-4386
> URL: https://issues.apache.org/jira/browse/PHOENIX-4386
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Thomas D'Silva
>Assignee: Thomas D'Silva
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4386-addendum-0.98.patch, 
> PHOENIX-4386-addendum-master.patch, PHOENIX-4386.patch, 
> PHOENIX-4386_5.x-HBase-2.0.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4386) Calculate the estimatedSize of MutationState using Map<TableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations

2018-02-20 Thread Rajeshbabu Chintaguntla (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajeshbabu Chintaguntla updated PHOENIX-4386:
-
Attachment: PHOENIX-4386_5.x-HBase-2.0.patch

> Calculate the estimatedSize of MutationState using Map Map> mutations
> ---
>
> Key: PHOENIX-4386
> URL: https://issues.apache.org/jira/browse/PHOENIX-4386
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Thomas D'Silva
>Assignee: Thomas D'Silva
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4386-addendum-0.98.patch, 
> PHOENIX-4386-addendum-master.patch, PHOENIX-4386.patch, 
> PHOENIX-4386_5.x-HBase-2.0.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid

2018-02-20 Thread Ohad Shacham (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369840#comment-16369840
 ] 

Ohad Shacham commented on PHOENIX-3860:
---

[~jamestaylor], another issue, in addition to the one above. Omid has an 
internal low water mark that is used to bound the size of the transaction 
manager's data structure (for conflict detection) and also to mark the point 
where the garbage collector can delete entries below (while keeping below this 
point only one version per key, if exists).

When a transaction tries to commit with a begin id smaller than this low water 
mark the transaction manager returns abort from two reasons: First, information 
in the conflict detection data structure that might conflict with this 
transaction might be missing. And second, the garbage collector might deleted 
information that was supposed to be read by the transaction (This hurts Omid's 
Opacity).

 

If during the index creation the low water exceeds the fence id then there 
might be a case that the scan of the data table will miss some of the 
information that needs to be written to the index.

My question is how the index creation works. Do you create an index using a 
transaction and commit when you finish populate all the data table info to the 
index? What happened if the commit fails? You drop the index table, take a new 
fence and start the process again? 

If this is the case and the index is not valid until the transaction 
successfully commit (which it should be invalid) then we can do it the same 
with Omid and this will overcome this issue.

The only difference from regular transaction is that we will directly write the 
fence id to the shadow cell in order to remove the need of saving all the 
mutations in the client (for future shadow cells update). This update is OK 
only assuming that the index is not valid until all the data table info is 
populated and that if the commit fails then the table drops and a new one is 
created.

 

> Implement TAL functionality for Omid
> 
>
> Key: PHOENIX-3860
> URL: https://issues.apache.org/jira/browse/PHOENIX-3860
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Implement TAL functionality for Omid in order to be able to use Omid as 
> Phoenix's transaction processing engine. 
> To support the integration jira [OMID-82] was opened that encapsulates all 
> Omid related development for Phoenix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid

2018-02-20 Thread Ohad Shacham (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369798#comment-16369798
 ] 

Ohad Shacham commented on PHOENIX-3860:
---

This will require writing/augmenting  a coprocessor that will add Omid's shadow 
cells to the index updates once the transaction successfully committed.

Originally I meant to the index creation from a non empty data table and not to 
the index update. For global index, the population is done at the client side 
and for local it is done at the server side.

Could you please refer me to this code? We also have problem with the shadow 
cells in this case.

> Implement TAL functionality for Omid
> 
>
> Key: PHOENIX-3860
> URL: https://issues.apache.org/jira/browse/PHOENIX-3860
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Ohad Shacham
>Assignee: Ohad Shacham
>Priority: Major
>
> Implement TAL functionality for Omid in order to be able to use Omid as 
> Phoenix's transaction processing engine. 
> To support the integration jira [OMID-82] was opened that encapsulates all 
> Omid related development for Phoenix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)