[jira] [Assigned] (PHOENIX-3430) Optimizer not using all columns from secondary index

[Swaroopa Kadam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam reassigned PHOENIX-3430:
---

Assignee: Swaroopa Kadam

> Optimizer not using all columns from secondary index
> 
>
> Key: PHOENIX-3430
> URL: https://issues.apache.org/jira/browse/PHOENIX-3430
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Zhenhua Xu
>Assignee: Swaroopa Kadam
>Priority: Major
>
>  Setup =
> DROP TABLE IF EXISTS TEST.TEMP;
> CREATE TABLE TEST.TEMP (
> ORGANIZATION_ID CHAR(15) NOT NULL,
> NETWORK_ID CHAR(15) NOT NULL,
> ENTITY_ID CHAR(15) NOT NULL,
> SCORE DOUBLE
> CONSTRAINT TOP_ENTITY_PK PRIMARY KEY (
> ORGANIZATION_ID,
> NETWORK_ID,
> ENTITY_ID
> )
> ) VERSIONS=1;
> CREATE INDEX IF NOT EXISTS TEMP_INDEX ON TEST.TEMP (ORGANIZATION_ID, 
> NETWORK_ID, SCORE DESC, ENTITY_ID DESC);
> EXPLAIN
> SELECT entity_id, MAX(score) FROM TEST.TEMP
> WHERE organization_id = 'organization_id'
>   AND (network_id = 'network_id' OR network_id='network_id1')
>   AND ((score = 9.0 AND entity_id < 'entity_id') OR score < 9.0)
> GROUP BY entity_id
> ORDER BY MAX(score) DESC, entity_id DESC
> LIMIT 100;
> === Execution Plan ===
> -CLIENT 1-CHUNK PARALLEL 1-WAY SKIP SCAN ON 2 KEYS OVER TEST.TEMP_INDEX 
> ['organization_id','network_id '] - ['organization_id','network_id1']
> --SERVER FILTER BY FIRST KEY ONLY AND ((TO_DOUBLE("SCORE") = 9.0 AND 
> "ENTITY_ID" < 'entity_id') OR TO_DOUBLE("SCORE") < 9.0)
> --SERVER AGGREGATE INTO DISTINCT ROWS BY ["ENTITY_ID"]
> -CLIENT MERGE SORT
> -CLIENT TOP 100 ROWS SORTED BY [MAX(TO_DOUBLE("SCORE")) DESC, "ENTITY_ID" 
> DESC]
> The execution plan shows a server-side skip scans using only the first 2 
> columns in the TEMP_INDEX secondary index. It could have used the SCORE and 
> ENTITY_ID columns  to speed up server side scans also.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Board report due in ~1 week

[Thomas D'Silva]

Should we use 2.0.5 ( the latest released HBase version )? If PHOENIX-5250
 is a blocker we will
need to fix it before we can release 5.0.1 (or 5.1).

These are the list of bugs that have a fix version of 5.1 :
https://issues.apache.org/jira/issues/?jql=project%20%3D%20PHOENIX%20AND%20fixVersion%20%3D%205.1.0%20AND%20issuetype%20%20%3D%20Bug
We should figure out which of these we want to cherrypick for the 5.0.1
release.


On Mon, Apr 29, 2019 at 11:34 PM Jaanai Zhang 
wrote:

> I will try doing release 5.0.1. Tow things need to confirm:
>
> 1.  According to the thread[1],  a new Phoenix 5.0.1 will support HBase
> 2.0.2, right? As far as I know, some improvements of HBase 2.0.2 will cause
> some critical issues, for example,
> https://issues.apache.org/jira/browse/PHOENIX-5250 (This issue from our
> production environments, we had used HBase 2.0.2 in Phoenix 5.0).
>
> 2.  We should have a clear JIRA list for a 5.0.1, I am not sure what JIRAs
> should go into it, only critical and blocked JIRAs apply for a 5.0.1?  If I
> know what priority need to pick, I will collect them.
>
>
> 
>Jaanai Zhang
>Best regards!
>
>
>
> Thomas D'Silva  于2019年4月30日周二 上午10:06写道：
>
> > Jaanai,
> >
> > We are waiting for a few omid bug fixes to do the 4.15/5.1 release that
> > will have splittable system catalog and the omid integration.
> > Are you interested in doing a 5.0.1 release that has the HBase 2.0.x
> > compatibility fixes that were discussed in a previous thread[1]?
> >
> > The steps to create a RC are straightforward and documented here:
> > https://phoenix.apache.org/release.html.
> > The main thing you need to do is to add your code signing key to
> > https://dist.apache.org/repos/dist/dev/phoenix/KEYS (follow the steps at
> > the start of that file)
> > and then commit using svn. Then you can follow the rest of the steps
> listed
> > in "How to do a release"
> >
> > Thanks,
> > Thomas
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/99fcc737d7a8f82ddffb1b34a64f7099f7909900b8bea36dd6afca16@%3Cdev.phoenix.apache.org%3E
> >
> > On Mon, Apr 29, 2019 at 6:33 PM Jaanai Zhang 
> > wrote:
> >
> > > I would like to volunteer for a new 5.x release if someone can guide me
> > > release process. Thanks.
> > >
> > > 
> > >Jaanai Zhang
> > >Best regards!
> > >
> > >
> > >
> > > Josh Elser  于2019年4月30日周二 上午12:39写道：
> > >
> > > > Hiya folks,
> > > >
> > > > It's about that time for another board report. Please reply here with
> > > > anything of merit that you think the board might find
> > > > interesting/useful. As a reminder, they board is typically more
> > > > concerned with high-level project/community details than the
> > > > nuts-and-bolts of the code changes for the project.
> > > >
> > > > On my radar already is...
> > > >
> > > > * Multiple new committers and PMC'ers (thanks so much to the folks
> who
> > > > have been driving votes!)
> > > > * NoSQL day in May
> > > > * 4.14.2 in vote
> > > > * Need for a new 5.x.y release (if there are no volunteers, I may
> have
> > > > to find the time to make this happen. It's been too long)
> > > >
> > > > Thanks!
> > > >
> > > > - Josh
> > > >
> > >
> >
>

[jira] [Updated] (PHOENIX-4940) IndexTool should be able to rebuild tenant-owned indexes

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-4940:

Fix Version/s: (was: 5.1)
   5.1.0

> IndexTool should be able to rebuild tenant-owned indexes
> 
>
> Key: PHOENIX-4940
> URL: https://issues.apache.org/jira/browse/PHOENIX-4940
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.14.0, 5.0.0
>Reporter: Geoffrey Jacoby
>Assignee: Gokcen Iskender
>Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-4940-4.x.patch, PHOENIX-4940-addendum.patch
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> IndexTool uses global connections to lookup the indexes which it's asked to 
> rebuild, which means that it won't be able to see indexes owned by tenant 
> views. We should add an optional tenantId parameter to it that will use a 
> tenant connection (and potentially our MapReduce framework's tenant 
> connection support) to allow for rebuilding those indexes as well. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4781) Phoenix client project's jar naming convention causes maven-deploy-plugin to fail

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-4781:

Fix Version/s: (was: 5.1)
   5.1.0

> Phoenix client project's jar naming convention causes maven-deploy-plugin to 
> fail
> -
>
> Key: PHOENIX-4781
> URL: https://issues.apache.org/jira/browse/PHOENIX-4781
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 5.0.0, 4.14.1
>Reporter: Karan Mehta
>Assignee: Karan Mehta
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2
>
> Attachments: PHOENIX-4781.001.patch, PHOENIX-4781.002.patch, 
> PHOENIX-4781.4.x-HBase-1.4.v3.patch, PHOENIX-4781.4.x-HBase-1.4.v4.patch, 
> PHOENIX-4781.4.x-HBase-1.4.v5.patch, PHOENIX-4781.addendum.patch
>
>
> `maven-deploy-plugin` is used for deploying built artifacts to repository 
> provided by `distributionManagement` tag. The name of files that need to be 
> uploaded are either derived from pom file of the project or it generates an 
> temporary one on its own.
> For `phoenix-client` project, we essentially create a shaded uber jar that 
> contains all dependencies and provide the project pom file for the plugin to 
> work. `maven-jar-plugin` is disabled for the project, hence the shade plugin 
> essentially packages the jar. The final name of the shaded jar is defined as 
> `phoenix-${project.version}\-client`, which is different from how the 
> standard maven convention based on pom file (artifact and group id) is 
> `phoenix-client-${project.version}`
> This causes `maven-deploy-plugin` to fail since it is unable to find any 
> artifacts to be published.
> `maven-install-plugin` works correctly and hence it installs correct jar in 
> local repo.
> The same is effective for `phoenix-pig` project as well. However we require 
> the require jar for that project in the repo. I am not even sure why we 
> create shaded jar for that project.
> I will put up a 3 liner patch for the same.
> Any thoughts? [~sergey.soldatov] [~elserj]
> Files before change (first col is size):
> {code:java}
> 103487701 Jun 13 22:47 
> phoenix-4.14.0-HBase-1.3-sfdc-1.0.14-SNAPSHOT-client.jar{code}
> Files after change (first col is size):
> {code:java}
> 3640 Jun 13 21:23 
> original-phoenix-client-4.14.0-HBase-1.3-sfdc-1.0.14-SNAPSHOT.jar
> 103487702 Jun 13 21:24 
> phoenix-client-4.14.0-HBase-1.3-sfdc-1.0.14-SNAPSHOT.jar{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4345) Error message for incorrect index is not accurate

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-4345:

Fix Version/s: (was: 5.1)
   5.1.0

> Error message for incorrect index is not accurate
> -
>
> Key: PHOENIX-4345
> URL: https://issues.apache.org/jira/browse/PHOENIX-4345
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Ethan Wang
>Assignee: Xinyi Yan
>Priority: Trivial
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-4345-4.x-HBase-1.4.patch, PHOENIX-4345.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Error message for incorrect index is not accurate. it shows "table 
> undefined". rather, should be index undefined.
> Table name: PERSON
> Index name: LOCAL_ADDRESS
> 0: jdbc:phoenix:localhost:2181:/hbase> ALTER INDEX LOCAL_ADDRESSX ON PERSON 
> rebuild;
> Error: ERROR 1012 (42M03): Table undefined. tableName=LOCAL_ADDRESSX 
> (state=42M03,code=1012)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5066) The TimeZone is incorrectly used during writing or reading data

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5066:

Fix Version/s: (was: 5.1)
   5.1.0

> The TimeZone is incorrectly used during writing or reading data
> ---
>
> Key: PHOENIX-5066
> URL: https://issues.apache.org/jira/browse/PHOENIX-5066
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.1
>Reporter: jaanai
>Assignee: jaanai
>Priority: Critical
> Fix For: 4.15.0, 5.1.0
>
> Attachments: DateTest.java
>
>
> We have two methods to write data when uses JDBC API.
> #1. Uses _the exceuteUpdate_ method to execute a string that is an upsert SQL.
> #2. Uses the _prepareStatement_ method to set some objects and execute.
> The _string_ data needs to convert to a new object by the schema information 
> of tables. we'll use some date formatters to convert string data to object 
> for Date/Time/Timestamp types when writes data and the formatters are used 
> when reads data as well.
>  
> *Uses default timezone test*
>  Writing 3 records by the different ways.
> {code:java}
> UPSERT INTO date_test VALUES (1,'2018-12-10 15:40:47','2018-12-10 
> 15:40:47','2018-12-10 15:40:47') 
> UPSERT INTO date_test VALUES (2,to_date('2018-12-10 
> 15:40:47'),to_time('2018-12-10 15:40:47'),to_timestamp('2018-12-10 15:40:47'))
> stmt.setInt(1, 3);stmt.setDate(2, date);stmt.setTime(3, 
> time);stmt.setTimestamp(4, ts);
> {code}
> Reading the table by the getObject(getDate/getTime/getTimestamp) methods.
> {code:java}
> 1 | 2018-12-10 | 23:45:07 | 2018-12-10 23:45:07.0 
> 2 | 2018-12-10 | 23:45:07 | 2018-12-10 23:45:07.0 
> 3 | 2018-12-10 | 15:45:07 | 2018-12-10 15:45:07.66 
> {code}
> Reading the table by the getString methods 
> {code:java}
> 1 | 2018-12-10 15:45:07.000 | 2018-12-10 15:45:07.000 | 2018-12-10 
> 15:45:07.000 
> 2 | 2018-12-10 15:45:07.000 | 2018-12-10 15:45:07.000 | 2018-12-10 
> 15:45:07.000 
> 3 | 2018-12-10 07:45:07.660 | 2018-12-10 07:45:07.660 | 2018-12-10 
> 07:45:07.660
> {code}
>  *Uses GMT+8 test*
>  Writing 3 records by the different ways.
> {code:java}
> UPSERT INTO date_test VALUES (1,'2018-12-10 15:40:47','2018-12-10 
> 15:40:47','2018-12-10 15:40:47')
> UPSERT INTO date_test VALUES (2,to_date('2018-12-10 
> 15:40:47'),to_time('2018-12-10 15:40:47'),to_timestamp('2018-12-10 15:40:47'))
> stmt.setInt(1, 3);stmt.setDate(2, date);stmt.setTime(3, 
> time);stmt.setTimestamp(4, ts);
> {code}
> Reading the table by the getObject(getDate/getTime/getTimestamp) methods.
> {code:java}
> 1 | 2018-12-10 | 23:40:47 | 2018-12-10 23:40:47.0 
> 2 | 2018-12-10 | 15:40:47 | 2018-12-10 15:40:47.0 
> 3 | 2018-12-10 | 15:40:47 | 2018-12-10 15:40:47.106 {code}
> Reading the table by the getString methods
> {code:java}
>  1 | 2018-12-10 23:40:47.000 | 2018-12-10 23:40:47.000 | 2018-12-10 
> 23:40:47.000
> 2 | 2018-12-10 15:40:47.000 | 2018-12-10 15:40:47.000 | 2018-12-10 
> 15:40:47.000
> 3 | 2018-12-10 15:40:47.106 | 2018-12-10 15:40:47.106 | 2018-12-10 
> 15:40:47.106
> {code}
>  
> _We_ have a historical problem,  we'll parse the string to 
> Date/Time/Timestamp objects with timezone in #1, which means the actual data 
> is going to be changed when stored in HBase table。



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-2265) Disallow creation of view over HBase table if PK not specified

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-2265:

Fix Version/s: (was: 5.1)
   5.1.0

> Disallow creation of view over HBase table if PK not specified
> --
>
> Key: PHOENIX-2265
> URL: https://issues.apache.org/jira/browse/PHOENIX-2265
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Xinyi Yan
>Priority: Major
>  Labels: newbie
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-2265-4.x-HBase-1.4.patch, 
> PHOENIX-2265-master.patch, PHOENIX-2265.patch, PHOENIX-2265.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> We currently allow a Phoenix view to be defined over an HBase table without 
> specifying a primary key.
> To repro, create an HBase table in the HBase shell:
> {code}
>  create 'hb1', 'f1'
> {code}
> Then create a view in Phoenix:
> {code}
> create view "hb1"("f1".a varchar);
> {code}
> This should yield an error, as we haven't specified a primary key.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5073) Invalid PIndexState during rolling upgrade from 4.13 to 4.14

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5073:

Fix Version/s: (was: 5.1)
   5.1.0

> Invalid PIndexState during rolling upgrade from 4.13 to 4.14
> 
>
> Key: PHOENIX-5073
> URL: https://issues.apache.org/jira/browse/PHOENIX-5073
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Kiran Kumar Maturi
>Assignee: Kiran Kumar Maturi
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2
>
> Attachments: PHOENIX-5073-4.14-HBase-1.3.01.patch, 
> PHOENIX-5073-4.x-HBase-1.3.001.patch, PHOENIX-5073-4.x-HBase-1.3.002.patch, 
> PHOENIX-5073-4.x-HBase-1.3.003.1.patch, 
> PHOENIX-5073-4.x-HBase-1.3.003.2.patch, PHOENIX-5073-4.x-HBase-1.3.003.patch, 
> PHOENIX-5073-master-01.patch
>
>
> While doing a rolling upgrade from 4.13 to 4.14 we are seeing this exception. 
> {code:java}
> 2018-08-20 09:00:34,980 WARN  [pool-1-thread-1] workload.WriteWorkload - 
> java.util.concurrent.ExecutionException: java.sql.SQLException: 
> java.lang.IllegalArgumentException: Unable to PIndexState enum for serialized 
> value of 'w'
>     at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>     at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload.waitForBatches(WriteWorkload.java:233)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload.exec(WriteWorkload.java:183)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload.access$100(WriteWorkload.java:56)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload$1.run(WriteWorkload.java:159)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> Caused by: java.sql.SQLException: java.lang.IllegalArgumentException: Unable 
> to PIndexState enum for serialized value of 'w'
>     at 
> org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1322)
>     at 
> org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1284)
>     at 
> org.apache.phoenix.query.ConnectionQueryServicesImpl.getTable(ConnectionQueryServicesImpl.java:1501)
>     at 
> org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:581)
>     at 
> org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:504)
>     at 
> org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:496)
>     at 
> org.apache.phoenix.schema.MetaDataClient.updateCache(MetaDataClient.java:492)
>     at 
> org.apache.phoenix.execute.MutationState.validate(MutationState.java:780)
>     at 
> org.apache.phoenix.execute.MutationState.validateAll(MutationState.java:768)
>     at org.apache.phoenix.execute.MutationState.send(MutationState.java:980)
>     at org.apache.phoenix.execute.MutationState.send(MutationState.java:1469)
>     at 
> org.apache.phoenix.execute.MutationState.commit(MutationState.java:1301)
>     at 
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:539)
>     at 
> org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:536)
>     at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>     at 
> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:536)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:291)
>     at 
> org.apache.phoenix.pherf.workload.WriteWorkload$2.call(WriteWorkload.java:250)
>     ... 4 more
> Caused by: java.lang.IllegalArgumentException: Unable to PIndexState enum for 
> serialized value of 'w'
>     at 
> org.apache.phoenix.schema.PIndexState.fromSerializedValue(PIndexState.java:81)
>     at 
> org.apache.phoenix.schema.PTableImpl.createFromProto(PTableImpl.java:1222)
>     at 
> org.apache.phoenix.schema.PTableImpl.createFromProto(PTableImpl.java:1246)
>     at 
> org.apache.phoenix.coprocessor.MetaDataProtocol$MetaDataMutationResult.constructFromProto(MetaDataProtocol.java:330)
>     at 
> org.apache.phoenix.query.ConnectionQueryServicesImpl.metaDataCoprocessorExec(ConnectionQueryServicesImpl.java:1314){code}
>  
> Steps to reproduce.
>  # Start the server on 4.14
>  # Start load with both 4.13 and 4.14 clients
>  # 4.13 client will show the above error (it will only when Index state 
> transtition to PENDING_DISABLE , this state is not defined in 4.13) 
>  



--
This message was sent 

[jira] [Updated] (PHOENIX-5018) Index mutations created by UPSERT SELECT will have wrong timestamps

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5018:

Fix Version/s: (was: 5.1)
   5.1.0

> Index mutations created by UPSERT SELECT will have wrong timestamps
> ---
>
> Key: PHOENIX-5018
> URL: https://issues.apache.org/jira/browse/PHOENIX-5018
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0, 5.0.0
>Reporter: Geoffrey Jacoby
>Assignee: Kadir OZDEMIR
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2
>
> Attachments: PHOENIX-5018.4.x-HBase-1.3.001.patch, 
> PHOENIX-5018.4.x-HBase-1.3.002.patch, PHOENIX-5018.4.x-HBase-1.4.001.patch, 
> PHOENIX-5018.4.x-HBase-1.4.002.patch, PHOENIX-5018.master.001.patch, 
> PHOENIX-5018.master.002.patch, PHOENIX-5018.master.003.patch, 
> PHOENIX-5018.master.004.patch
>
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> When doing a full rebuild (or initial async build) of a local or global index 
> using IndexTool and PhoenixIndexImportDirectMapper, or doing a synchronous 
> initial build of a global index using the index create DDL, we generate the 
> index mutations by using an UPSERT SELECT query from the base table to the 
> index.
> The timestamps of the mutations use the default HBase behavior, which is to 
> take the current wall clock. However, the timestamp of an index KeyValue 
> should use the timestamp of the initial KeyValue in the base table.
> Having base table and index timestamps out of sync can cause all sorts of 
> weird side effects, such as if the base table has data with an expired TTL 
> that isn't expired in the index yet. Also inserting old mutations with new 
> timestamps may overwrite the data that has been newly overwritten by the 
> regular data path during index build, which would lead to data loss and 
> inconsistency issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5089) IndexScrutinyTool should be able to analyze tenant-owned indexes

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5089:

Fix Version/s: (was: 5.1)
   5.1.0

> IndexScrutinyTool should be able to analyze tenant-owned indexes
> 
>
> Key: PHOENIX-5089
> URL: https://issues.apache.org/jira/browse/PHOENIX-5089
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Geoffrey Jacoby
>Assignee: Gokcen Iskender
>Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5089-4x-1.4.patch, PHOENIX-5089.patch
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> IndexScrutiny uses global connections to lookup the indexes which it's asked 
> to analyze, which means that it won't be able to see indexes owned by tenant 
> views. We should add an optional tenantId parameter to it that will use a 
> tenant connection (and potentially our MapReduce framework's tenant 
> connection support) to allow for analyzing those indexes as well.
> This is similar to PHOENIX-4940 for the index rebuild tool.  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5168) IndexScrutinyTool cannot output to table when analyzing tenant-owned indexes

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5168:

Fix Version/s: (was: 5.1)
   5.1.0

> IndexScrutinyTool cannot output to table when analyzing tenant-owned indexes
> 
>
> Key: PHOENIX-5168
> URL: https://issues.apache.org/jira/browse/PHOENIX-5168
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Gokcen Iskender
>Assignee: Gokcen Iskender
>Priority: Major
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5168-4.x.patch, PHOENIX-5168.patch
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> IndexScrutiny has a new feature of using tenant connections (when tenant-Id 
> parameter is used) to lookup the indexes on tenant views.
> When tenant-id option is provided and output-format is set to TABLE, we get 
> an error that PHOENIX_SCRUTINY_TABLE cannot be created due to permission. 
> We should be able to output to a table when tenant-id option is used.
> Note that output-format set to FILE is supported.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5169) Query logger is still initialized for each query when the log level is off

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5169:

Fix Version/s: (was: 5.1)
   5.1.0

> Query logger is still initialized for each query when the log level is off
> --
>
> Key: PHOENIX-5169
> URL: https://issues.apache.org/jira/browse/PHOENIX-5169
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: jaanai
>Assignee: jaanai
>Priority: Major
> Fix For: 5.1.0, 4.14.2
>
> Attachments: PHOENIX-5169-master-v2.patch, 
> PHOENIX-5169-master-v3.patch, PHOENIX-5169-master-v4.patch, 
> PHOENIX-5169-master.patch, image-2019-02-28-10-05-00-518.png
>
>
> we will still invoke createQueryLogger in PhoenixStatement for each query 
> when query logger level is OFF, which has significant throughput impacts 
> under multiple threads.
> The below is jstack with the concurrent query:
> !https://gw.alicdn.com/tfscom/TB1HC3bI4TpK1RjSZFMXXbG_VXa.png|width=500,height=400!
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5132) View indexes with different owners but of the same base table can be assigned same ViewIndexId

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5132:

Fix Version/s: (was: 5.1)
   5.1.0

> View indexes with different owners but of the same base table can be assigned 
> same ViewIndexId
> --
>
> Key: PHOENIX-5132
> URL: https://issues.apache.org/jira/browse/PHOENIX-5132
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.1
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Critical
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5132-4.x-HBase-1.4.patch, 
> PHOENIX-5132-4.x-HBase-1.4.v2.patch, PHOENIX-5132-repro.patch
>
>
> All indexes on views for a particular base table are stored in the same 
> physical HBase table. Phoenix distinguishes them by prepending each row key 
> with an encoded short or long integer called a ViewIndexId. 
> The ViewIndexId is generated by using a sequence to guarantee that each view 
> index id is unique. Unfortunately, the sequence used follows a convention of 
> [SaltByte, Tenant, Schema, BaseTable] for its key, which means that there's a 
> separate sequence for each tenant that owns an index in the view index table. 
> (See MetaDataUtil.getViewIndexSequenceKey) Since all the sequences start at 
> the same value, collisions are not only possible but likely. 
> I've written a test that confirms the ViewIndexId collision. This means it's 
> very likely that query results using one view index could mistakenly include 
> rows from another index, but I haven't confirmed this. 
> All view indexes for a base table, regardless of whether globally or 
> tenant-owned, should use the same sequence. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5188) IndexedKeyValue should populate KeyValue fields

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5188:

Fix Version/s: (was: 5.1)
   5.1.0

> IndexedKeyValue should populate KeyValue fields
> ---
>
> Key: PHOENIX-5188
> URL: https://issues.apache.org/jira/browse/PHOENIX-5188
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 5.0.0, 4.14.1
>Reporter: Geoffrey Jacoby
>Assignee: Geoffrey Jacoby
>Priority: Major
> Fix For: 4.15.0, 5.1.0, 4.14.2
>
> Attachments: PHOENIX-5188-4.x-HBase-1.4..addendum.patch, 
> PHOENIX-5188-4.x-HBase-1.4.patch, PHOENIX-5188.patch
>
>
> IndexedKeyValue subclasses the HBase KeyValue class, which has three primary 
> fields: bytes, offset, and length. These fields aren't populated by 
> IndexedKeyValue because it's concerned with index mutations, and has its own 
> fields that its own methods use. 
> However, KeyValue and its Cell interface have quite a few methods that assume 
> these fields are populated, and the HBase-level factory methods generally 
> ensure they're populated. Phoenix code should do the same, to maintain the 
> polymorphic contract. This is important in cases like custom 
> ReplicationEndpoints where HBase-level code may be iterating over WALEdits 
> that contain both KeyValues and IndexKeyValues and may need to interrogate 
> their contents. 
> Since the index mutation has a row key, this is straightforward. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5251) Avoid taking explicit lock by using AtomicReference in PhoenixAccessController class

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5251:

Fix Version/s: (was: 5.1)
   5.1.0

> Avoid taking explicit lock by using AtomicReference in 
> PhoenixAccessController class
> 
>
> Key: PHOENIX-5251
> URL: https://issues.apache.org/jira/browse/PHOENIX-5251
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Swaroopa Kadam
>Assignee: Swaroopa Kadam
>Priority: Minor
> Fix For: 4.15.0, 5.1.0
>
> Attachments: PHOENIX-5251.4.x-HBase-1.3.v1.patch, 
> PHOENIX-5251.master.v1.patch
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> By [~elserj] on PHOENIX-5070
> If we want to avoid taking an explicit lock, what about using AtomicReference 
> instead? Can we spin out another Jira issue to fix that?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5250) The accumulated wal files can't be cleaned

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva updated PHOENIX-5250:

Fix Version/s: 5.1.0

> The accumulated wal files can't be cleaned
> --
>
> Key: PHOENIX-5250
> URL: https://issues.apache.org/jira/browse/PHOENIX-5250
> Project: Phoenix
>  Issue Type: Bug
>Reporter: jaanai
>Priority: Blocker
> Fix For: 5.1.0
>
> Attachments: image-2019-04-19-21-31-27-888.png
>
>
> Because of the modification of HBASE-20781,  the faked WALEdits won't be 
> filtered, all WALEdits will be saved into Memstore with a status that 
> inMemstore is true(uses WAL->append method).
> !image-2019-04-19-21-31-27-888.png|width=755,height=310!
> The family array of IndexedKeyValue is WALEdit.METAFAMILY that is used to 
> describe a fake WALEdit, and it will put into Memstore with WALedits of data 
> table during sync global index.
> WAL files can't be cleaned, expect for resarting RS, Many WAL files will 
> decrease the percent of disk free.  
> !https://gw.alicdn.com/tfscom/TB1n3cDQVzqK1RjSZFCXXbbxVXa.png|width=422,height=159!
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New Phoenix committer: Mihir Monani

[Geoffrey Jacoby]

Congratulations, Mihir!

On Tue, Apr 30, 2019 at 6:01 PM Ankit Singhal 
wrote:

> Congratulations and Welcome Mihir !!
>
> On Tue, Apr 30, 2019 at 5:49 PM Andrew Purtell 
> wrote:
>
> > Congratulations Mihir.
> >
> > On Sat, Apr 27, 2019 at 5:42 PM Thomas D'Silva 
> wrote:
> >
> > > On behalf of the Apache Phoenix PMC, I am pleased to announce that
> > > Mihir Monani has accepted our invitation to become a committer.
> > > Mihir has done some nice work fixing several bugs related to
> indexing[1].
> > >
> > > Please welcome him to the Apache Phoenix team.
> > >
> > > Thanks,
> > > Thomas
> > >
> > > [1]
> > >
> > >
> >
> https://issues.apache.org/jira/browse/PHOENIX-5199?jql=project%20%3D%20PHOENIX%20AND%20assignee%3D%22mihir6692%22%20AND%20status%3DResolved
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> >- A23, Crosstalk
> >
>

[jira] [Updated] (PHOENIX-5262) Wrong Result on Salted table with some Variable Length PKs

[Daniel Wong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Wong updated PHOENIX-5262:
-
Attachment: PHOENIX-5262v3.patch

> Wrong Result on Salted table with some Variable Length PKs
> --
>
> Key: PHOENIX-5262
> URL: https://issues.apache.org/jira/browse/PHOENIX-5262
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Daniel Wong
>Assignee: Daniel Wong
>Priority: Major
> Attachments: PHOENIX-5262.patch, PHOENIX-5262v2.patch, 
> PHOENIX-5262v3.patch
>
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ANNOUNCE] New Phoenix committer: Mihir Monani

[Ankit Singhal]

Congratulations and Welcome Mihir !!

On Tue, Apr 30, 2019 at 5:49 PM Andrew Purtell  wrote:

> Congratulations Mihir.
>
> On Sat, Apr 27, 2019 at 5:42 PM Thomas D'Silva  wrote:
>
> > On behalf of the Apache Phoenix PMC, I am pleased to announce that
> > Mihir Monani has accepted our invitation to become a committer.
> > Mihir has done some nice work fixing several bugs related to indexing[1].
> >
> > Please welcome him to the Apache Phoenix team.
> >
> > Thanks,
> > Thomas
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/browse/PHOENIX-5199?jql=project%20%3D%20PHOENIX%20AND%20assignee%3D%22mihir6692%22%20AND%20status%3DResolved
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

Re: [ANNOUNCE] New Phoenix committer: Mihir Monani

[Andrew Purtell]

Congratulations Mihir.

On Sat, Apr 27, 2019 at 5:42 PM Thomas D'Silva  wrote:

> On behalf of the Apache Phoenix PMC, I am pleased to announce that
> Mihir Monani has accepted our invitation to become a committer.
> Mihir has done some nice work fixing several bugs related to indexing[1].
>
> Please welcome him to the Apache Phoenix team.
>
> Thanks,
> Thomas
>
> [1]
>
> https://issues.apache.org/jira/browse/PHOENIX-5199?jql=project%20%3D%20PHOENIX%20AND%20assignee%3D%22mihir6692%22%20AND%20status%3DResolved
>


-- 
Best regards,
Andrew

Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
   - A23, Crosstalk

[jira] [Assigned] (PHOENIX-1955) Phoenix create table with salt_bucket,then csvload data,index table is null

[Swaroopa Kadam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam reassigned PHOENIX-1955:
---

Assignee: Swaroopa Kadam

> Phoenix create table with salt_bucket,then csvload data,index table is null
> ---
>
> Key: PHOENIX-1955
> URL: https://issues.apache.org/jira/browse/PHOENIX-1955
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.0.0
> Environment: redhat6.4,hbase1.0.0,hadoop2.5.0
>Reporter: Alisa
>Assignee: Swaroopa Kadam
>Priority: Major
>  Labels: vierfy
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> The test steps is as follows:
> 1.create table use salt_bucket ，then create local index on the table
> 2.use phoenix csvload to load data into table and index table
> 3.load complete,select count(1) from index,index table is null,but scan index 
> table from hbase shell,it is not null



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (PHOENIX-2882) NPE during View creation for table with secondary index

[Swaroopa Kadam (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Swaroopa Kadam reassigned PHOENIX-2882:
---

Assignee: Swaroopa Kadam

> NPE during View creation for table with secondary index 
> 
>
> Key: PHOENIX-2882
> URL: https://issues.apache.org/jira/browse/PHOENIX-2882
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergey Soldatov
>Assignee: Swaroopa Kadam
>Priority: Major
>
> A simple test case:
> {noformat}
> create table test (id integer primary key, i1 integer, i2 integer);
> create index i1 on test (i1);
> create view v1 as select * from test where i2 <10;
> {noformat}
> the thrown exception:
> {noformat}
> org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): 
> Undefined column. columnName=0:I2
>   at org.apache.phoenix.schema.PTableImpl.getColumn(PTableImpl.java:692)
>   at 
> org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:444)
>   at 
> org.apache.phoenix.compile.ExpressionCompiler.resolveColumn(ExpressionCompiler.java:366)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.resolveColumn(WhereCompiler.java:181)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:169)
>   at 
> org.apache.phoenix.compile.WhereCompiler$WhereExpressionCompiler.visit(WhereCompiler.java:156)
>   at 
> org.apache.phoenix.parse.ColumnParseNode.accept(ColumnParseNode.java:56)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at org.apache.phoenix.parse.CastParseNode.accept(CastParseNode.java:60)
>   at 
> org.apache.phoenix.parse.CompoundParseNode.acceptChildren(CompoundParseNode.java:64)
>   at 
> org.apache.phoenix.parse.ComparisonParseNode.accept(ComparisonParseNode.java:45)
>   at 
> org.apache.phoenix.compile.WhereCompiler.compile(WhereCompiler.java:86)
>   at 
> org.apache.phoenix.util.IndexUtil.rewriteViewStatement(IndexUtil.java:494)
>   at 
> org.apache.phoenix.schema.MetaDataClient.addIndexesFromPhysicalTable(MetaDataClient.java:739)
>   at 
> org.apache.phoenix.schema.MetaDataClient.addTableToCache(MetaDataClient.java:3418)
>   at 
> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2279)
>   at 
> org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:866)
>   at 
> org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:343)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:331)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:330)
> {noformat}
> View created, but any select using view cause a similar exception



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5262) Wrong Result on Salted table with some Variable Length PKs

[Daniel Wong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Wong updated PHOENIX-5262:
-
Summary: Wrong Result on Salted table with some Variable Length PKs  (was: 
Wrong Result on Salted table with Varbinary PK)

> Wrong Result on Salted table with some Variable Length PKs
> --
>
> Key: PHOENIX-5262
> URL: https://issues.apache.org/jira/browse/PHOENIX-5262
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Daniel Wong
>Assignee: Daniel Wong
>Priority: Major
> Attachments: PHOENIX-5262.patch, PHOENIX-5262v2.patch
>
>  Time Spent: 4h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5265) Phoenix Test should use gold files for result comparison instead of using hard-corded comparison.

[Bin Shi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bin Shi updated PHOENIX-5265:
-
Description: 
Currently, in Phoenix Test, the comparison of the returned result and the 
expected result is always hard-coded, which is especially widely used by E2E 
and Integration test for comparing the query result, including the result of 
EXPLAIN query plan. This has significantly impaired the productivity and 
efficiency of workin on Phoenix.

The right approach is, each test case should write the generated result to a 
file under a new directory, then compares this file with a gold file under 
"gold" directory. After we make some code changes and make sure the change is 
correct by manually verifying several specific test cases, we can safely 
replace the whole "gold" directory with the directory which contains all new 
generated test files during test running. In this way, we can easily rebase all 
the tests.

For example, in BaseStatsCollectorIT.java, we verify the estimated row size in 
the returned result of EXPLAIN query plan, the row size is decided by many 
factors, like the column is encoded or not, it is mutable or not, it uses 
transaction provider or not, it uses TEPHRA or OMID as transaction provider, 
etc. The code snippet "testWithMultiCF" shows a typical test case. The 
comparisons of the result result and the expected result are hard-coded in 
those asserts. Now imagine, if we change the way collecting stats or we change 
column encoding scheme or we change the cell storage format for TEPHRA or OMID, 
which is very likely to happen, then we need to manually change all those 
hard-coded comparison everywhere, and it isn't trivial to re-calculate all 
expected row sizes with the different conditions . Today you might need one or 
two weeks to rebase all the tests, which is a huge waste. We should use "gold" 
files here, so that we can rebase the test very easily.

 BTW, the new generated test result files and gold files should be in XML or 
JSON. The result of "EXPLAN" query should be in XML or JSON too, because the 
file format matches the structure of a query plan.
{code:java}
// code placeholder
@Test
public void testWithMultiCF() throws Exception {
int nRows = 20;
Connection conn = getConnection(0);
PreparedStatement stmt;
conn.createStatement().execute(
"CREATE TABLE " + fullTableName
+ "(k VARCHAR PRIMARY KEY, a.v INTEGER, b.v INTEGER, c.v 
INTEGER NULL, d.v INTEGER NULL) "
+ tableDDLOptions );
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + " VALUES(?,?, 
?, ?, ?)");
byte[] val = new byte[250];
for (int i = 0; i < nRows; i++) {
stmt.setString(1, Character.toString((char)('a' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.setInt(4, i);
stmt.setInt(5, i);
stmt.executeUpdate();
}
conn.commit();
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + "(k, c.v, 
d.v) VALUES(?,?,?)");
for (int i = 0; i < 5; i++) {
stmt.setString(1, Character.toString((char)('a' + 'z' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.executeUpdate();
}
conn.commit();

ResultSet rs;
String actualExplainPlan;
collectStatistics(conn, fullTableName);
List keyRanges = getAllSplits(conn, fullTableName);
assertEquals(26, keyRanges.size());
rs = conn.createStatement().executeQuery("EXPLAIN SELECT * FROM " + 
fullTableName);
actualExplainPlan = QueryUtil.getExplainPlan(rs);
assertEquals(
"CLIENT 26-CHUNK 25 ROWS " + (columnEncoded ? ( mutable ? "12530" : 
"14190" ) : 
(TransactionFactory.Provider.OMID.name().equals(transactionProvider)) ? "25320" 
: "12420") +
" BYTES PARALLEL 1-WAY FULL SCAN OVER " + physicalTableName,
actualExplainPlan);

ConnectionQueryServices services = 
conn.unwrap(PhoenixConnection.class).getQueryServices();
List regions = 
services.getAllTableRegions(Bytes.toBytes(physicalTableName));
assertEquals(1, regions.size());

collectStatistics(conn, fullTableName, Long.toString(1000));
keyRanges = getAllSplits(conn, fullTableName);
boolean oneCellPerColFamilyStorageScheme = !mutable && columnEncoded;
boolean hasShadowCells = 
TransactionFactory.Provider.OMID.name().equals(transactionProvider);
assertEquals(oneCellPerColFamilyStorageScheme ? 14 : hasShadowCells ? 24 : 
13, keyRanges.size());

rs = conn
.createStatement()
.executeQuery(
"SELECT 
COLUMN_FAMILY,SUM(GUIDE_POSTS_ROW_COUNT),SUM(GUIDE_POSTS_WIDTH),COUNT(*) from 
\"SYSTEM\".STATS where PHYSICAL_NAME = '"
+ physicalTableName + "' GROUP BY COLUMN_FAMILY 
ORDER BY COLUMN_FAMILY");

assertTrue(rs.next());

[jira] [Updated] (PHOENIX-5265) Phoenix Test should use gold files for result comparison instead of using hard-corded comparison.

[Bin Shi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bin Shi updated PHOENIX-5265:
-
Description: 
Currently, in Phoenix Test, the comparison of the returned result and the 
expected result is always hard-coded, which is especially widely used by E2E 
and Integration test for comparing the query result, including the result of 
EXPLAIN query plan. This has significantly impaired the productivity and 
efficiency of workin on Phoenix.

The right approach is, each test case should write the generated result to a 
file under a new directory, then compares this file with a gold file under 
"gold" directory. After we make some code changes and make sure the change is 
correct by manually verifying several specific test cases, we can safely 
replace the whole "gold" directory with the directory which contains all new 
generated test files during test running. In this way, we can easily rebase all 
the tests.

For example, in BaseStatsCollectorIT.java, we verify the estimated row size in 
the returned result of EXPLAIN query plan, the row size is decided by many 
factors, like the column is encoded or not, it is mutable or not, it uses 
transaction provider or not, it uses TEPHRA or OMID as transaction provider, 
etc. The code snippet "testWithMultiCF" shows a typical test case. The 
comparisons of the result result and the expected result are hard-coded in 
those asserts. Now imagine, if we change the way collecting stats or we change 
column encoding scheme or we change the cell storage format for TEPHRA or OMID, 
which is very likely to happen, then we need to manually change all those 
hard-coded comparison everywhere, and it isn't trivial to re-calculate all 
expected row sizes with the different conditions . Today you might need one or 
two weeks to rebase all the tests, which is a huge waste. We should use "gold" 
files here, so that we can rebase the test very easily.

 
{code:java}
// code placeholder
@Test
public void testWithMultiCF() throws Exception {
int nRows = 20;
Connection conn = getConnection(0);
PreparedStatement stmt;
conn.createStatement().execute(
"CREATE TABLE " + fullTableName
+ "(k VARCHAR PRIMARY KEY, a.v INTEGER, b.v INTEGER, c.v 
INTEGER NULL, d.v INTEGER NULL) "
+ tableDDLOptions );
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + " VALUES(?,?, 
?, ?, ?)");
byte[] val = new byte[250];
for (int i = 0; i < nRows; i++) {
stmt.setString(1, Character.toString((char)('a' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.setInt(4, i);
stmt.setInt(5, i);
stmt.executeUpdate();
}
conn.commit();
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + "(k, c.v, 
d.v) VALUES(?,?,?)");
for (int i = 0; i < 5; i++) {
stmt.setString(1, Character.toString((char)('a' + 'z' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.executeUpdate();
}
conn.commit();

ResultSet rs;
String actualExplainPlan;
collectStatistics(conn, fullTableName);
List keyRanges = getAllSplits(conn, fullTableName);
assertEquals(26, keyRanges.size());
rs = conn.createStatement().executeQuery("EXPLAIN SELECT * FROM " + 
fullTableName);
actualExplainPlan = QueryUtil.getExplainPlan(rs);
assertEquals(
"CLIENT 26-CHUNK 25 ROWS " + (columnEncoded ? ( mutable ? "12530" : 
"14190" ) : 
(TransactionFactory.Provider.OMID.name().equals(transactionProvider)) ? "25320" 
: "12420") +
" BYTES PARALLEL 1-WAY FULL SCAN OVER " + physicalTableName,
actualExplainPlan);

ConnectionQueryServices services = 
conn.unwrap(PhoenixConnection.class).getQueryServices();
List regions = 
services.getAllTableRegions(Bytes.toBytes(physicalTableName));
assertEquals(1, regions.size());

collectStatistics(conn, fullTableName, Long.toString(1000));
keyRanges = getAllSplits(conn, fullTableName);
boolean oneCellPerColFamilyStorageScheme = !mutable && columnEncoded;
boolean hasShadowCells = 
TransactionFactory.Provider.OMID.name().equals(transactionProvider);
assertEquals(oneCellPerColFamilyStorageScheme ? 14 : hasShadowCells ? 24 : 
13, keyRanges.size());

rs = conn
.createStatement()
.executeQuery(
"SELECT 
COLUMN_FAMILY,SUM(GUIDE_POSTS_ROW_COUNT),SUM(GUIDE_POSTS_WIDTH),COUNT(*) from 
\"SYSTEM\".STATS where PHYSICAL_NAME = '"
+ physicalTableName + "' GROUP BY COLUMN_FAMILY 
ORDER BY COLUMN_FAMILY");

assertTrue(rs.next());
assertEquals("A", rs.getString(1));
assertEquals(25, rs.getInt(2));
assertEquals(columnEncoded ? ( mutable ? 12530 : 14190 ) : hasShadowCells ? 
25320 : 12420, rs.getInt(3));

[jira] [Created] (PHOENIX-5265) Phoenix Test should use gold files for result comparison instead of using hard-corded comparison.

[Bin Shi (JIRA)]

Bin Shi created PHOENIX-5265:


 Summary: Phoenix Test should use gold files for result comparison 
instead of using hard-corded comparison.
 Key: PHOENIX-5265
 URL: https://issues.apache.org/jira/browse/PHOENIX-5265
 Project: Phoenix
  Issue Type: Improvement
 Environment: {code:java}
// code placeholder
@Test
public void testWithMultiCF() throws Exception {
int nRows = 20;
Connection conn = getConnection(0);
PreparedStatement stmt;
conn.createStatement().execute(
"CREATE TABLE " + fullTableName
+ "(k VARCHAR PRIMARY KEY, a.v INTEGER, b.v INTEGER, c.v 
INTEGER NULL, d.v INTEGER NULL) "
+ tableDDLOptions );
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + " VALUES(?,?, 
?, ?, ?)");
byte[] val = new byte[250];
for (int i = 0; i < nRows; i++) {
stmt.setString(1, Character.toString((char)('a' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.setInt(4, i);
stmt.setInt(5, i);
stmt.executeUpdate();
}
conn.commit();
stmt = conn.prepareStatement("UPSERT INTO " + fullTableName + "(k, c.v, 
d.v) VALUES(?,?,?)");
for (int i = 0; i < 5; i++) {
stmt.setString(1, Character.toString((char)('a' + 'z' + i)) + 
Bytes.toString(val));
stmt.setInt(2, i);
stmt.setInt(3, i);
stmt.executeUpdate();
}
conn.commit();

ResultSet rs;
String actualExplainPlan;
collectStatistics(conn, fullTableName);
List keyRanges = getAllSplits(conn, fullTableName);
assertEquals(26, keyRanges.size());
rs = conn.createStatement().executeQuery("EXPLAIN SELECT * FROM " + 
fullTableName);
actualExplainPlan = QueryUtil.getExplainPlan(rs);
assertEquals(
"CLIENT 26-CHUNK 25 ROWS " + (columnEncoded ? ( mutable ? "12530" : 
"14190" ) : 
(TransactionFactory.Provider.OMID.name().equals(transactionProvider)) ? "25320" 
: "12420") +
" BYTES PARALLEL 1-WAY FULL SCAN OVER " + physicalTableName,
actualExplainPlan);

ConnectionQueryServices services = 
conn.unwrap(PhoenixConnection.class).getQueryServices();
List regions = 
services.getAllTableRegions(Bytes.toBytes(physicalTableName));
assertEquals(1, regions.size());

collectStatistics(conn, fullTableName, Long.toString(1000));
keyRanges = getAllSplits(conn, fullTableName);
boolean oneCellPerColFamilyStorageScheme = !mutable && columnEncoded;
boolean hasShadowCells = 
TransactionFactory.Provider.OMID.name().equals(transactionProvider);
assertEquals(oneCellPerColFamilyStorageScheme ? 14 : hasShadowCells ? 24 : 
13, keyRanges.size());

rs = conn
.createStatement()
.executeQuery(
"SELECT 
COLUMN_FAMILY,SUM(GUIDE_POSTS_ROW_COUNT),SUM(GUIDE_POSTS_WIDTH),COUNT(*) from 
\"SYSTEM\".STATS where PHYSICAL_NAME = '"
+ physicalTableName + "' GROUP BY COLUMN_FAMILY 
ORDER BY COLUMN_FAMILY");

assertTrue(rs.next());
assertEquals("A", rs.getString(1));
assertEquals(25, rs.getInt(2));
assertEquals(columnEncoded ? ( mutable ? 12530 : 14190 ) : hasShadowCells ? 
25320 : 12420, rs.getInt(3));
assertEquals(oneCellPerColFamilyStorageScheme ? 13 : hasShadowCells ? 23 : 
12, rs.getInt(4));

assertTrue(rs.next());
assertEquals("B", rs.getString(1));
assertEquals(oneCellPerColFamilyStorageScheme ? 25 : 20, rs.getInt(2));
assertEquals(columnEncoded ? ( mutable ? 5600 : 7260 ) : hasShadowCells ? 
11260 : 5540, rs.getInt(3));
assertEquals(oneCellPerColFamilyStorageScheme ? 7 : hasShadowCells ? 10 : 
5, rs.getInt(4));

assertTrue(rs.next());
assertEquals("C", rs.getString(1));
assertEquals(25, rs.getInt(2));
assertEquals(columnEncoded ? ( mutable ? 7005 : 7280 ) : hasShadowCells ? 
14085 : 6930, rs.getInt(3));
assertEquals(hasShadowCells ? 13 : 7, rs.getInt(4));

assertTrue(rs.next());
assertEquals("D", rs.getString(1));
assertEquals(25, rs.getInt(2));
assertEquals(columnEncoded ? ( mutable ? 7005 : 7280 ) : hasShadowCells ? 
14085 : 6930, rs.getInt(3));
assertEquals(hasShadowCells ? 13 : 7, rs.getInt(4));

assertFalse(rs.next());

// Disable stats
conn.createStatement().execute("ALTER TABLE " + fullTableName + 
" SET " + PhoenixDatabaseMetaData.GUIDE_POSTS_WIDTH + "=0");
collectStatistics(conn, fullTableName);
// Assert that there are no more guideposts
rs = conn.createStatement().executeQuery("SELECT count(1) FROM " + 
PhoenixDatabaseMetaData.SYSTEM_STATS_NAME + 
" WHERE " + PhoenixDatabaseMetaData.PHYSICAL_NAME + "='" + 
physicalTableName + "' AND " + PhoenixDatabaseMetaData.COLUMN_FAMILY + " IS NOT 
NULL");
assertTrue(rs.next());
assertEquals(0, rs.getLong(1));
assertFalse(rs.next());
rs = 

[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool

[Prashant Vithani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prashant Vithani updated PHOENIX-5258:
--
Summary: Add support to parse header from the input CSV file as input 
columns for CsvBulkLoadTool  (was: Add support for header in input CSV for 
CsvBulkLoadTool)

> Add support to parse header from the input CSV file as input columns for 
> CsvBulkLoadTool
> 
>
> Key: PHOENIX-5258
> URL: https://issues.apache.org/jira/browse/PHOENIX-5258
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Prashant Vithani
>Priority: Minor
>
> Currently, CsvBulkLoadTool does not support reading header from the input csv 
> and expects the content of the csv to match with the table schema. The 
> support for the header can be added to dynamically map the schema with the 
> header.
> The proposed solution is to introduce another option for the tool `–header`. 
> If this option is passed, the input columns list is constructed by reading 
> the first line of the input CSV file.
>  * If there is only one file, read the header from the first line and 
> generate the `ColumnInfo` list.
>  * If there are multiple files, read the header from all the files, and throw 
> an error if the headers across files do not match.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-5258) Add support for header in input CSV for CsvBulkLoadTool

[Prashant Vithani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prashant Vithani updated PHOENIX-5258:
--
Description: 
Currently, CsvBulkLoadTool does not support reading header from the input csv 
and expects the content of the csv to match with the table schema. The support 
for the header can be added to dynamically map the schema with the header.

The proposed solution is to introduce another option for the tool `–header`. If 
this option is passed, the input columns list is constructed by reading the 
first line of the input CSV file.
 * If there is only one file, read the header from the first line and generate 
the `ColumnInfo` list.
 * If there are multiple files, read the header from all the files, and throw 
an error if the headers across files do not match.

  was:Currently, CsvBulkLoadTool does not support reading header from the input 
csv and expects the content of the csv to match with the table schema. The 
support for the header can be added to dynamically map the schema with the 
header.


> Add support for header in input CSV for CsvBulkLoadTool
> ---
>
> Key: PHOENIX-5258
> URL: https://issues.apache.org/jira/browse/PHOENIX-5258
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Prashant Vithani
>Priority: Minor
>
> Currently, CsvBulkLoadTool does not support reading header from the input csv 
> and expects the content of the csv to match with the table schema. The 
> support for the header can be added to dynamically map the schema with the 
> header.
> The proposed solution is to introduce another option for the tool `–header`. 
> If this option is passed, the input columns list is constructed by reading 
> the first line of the input CSV file.
>  * If there is only one file, read the header from the first line and 
> generate the `ColumnInfo` list.
>  * If there are multiple files, read the header from all the files, and throw 
> an error if the headers across files do not match.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[DISCUSS] Next Phoenix 5.x release (was "Board report due in ~1 week")

[Josh Elser]

Let's leave the other thread for the board report, please. I've changed 
the subject as such.


Thanks for volunteering to be RM.

I'm of the opinion that we should just update to the latest HBase 2.0.x 
line. The compatibility assertions from HBase should make this a no-op 
for us to change versions. There are arguments, however, in both ways 
that we should use the earliest, non-breaking version of HBase.


As the RM, it is your prerogative to tell everyone else how you are 
tracking it :). Figure out what fixVersion you're using on Jira, and 
then either manage the list of open issues in there yourself, or push 
out issues and have folks put issues back in which they feel must be 
included.


Since this is your first time, I would prioritize an rc0 ASAP so that 
you can get comfortable with the process (even if we know that there are 
other fixes which must be made before the final release).


On 4/30/19 2:34 AM, Jaanai Zhang wrote:

I will try doing release 5.0.1. Tow things need to confirm:

1.  According to the thread[1],  a new Phoenix 5.0.1 will support HBase
2.0.2, right? As far as I know, some improvements of HBase 2.0.2 will cause
some critical issues, for example,
https://issues.apache.org/jira/browse/PHOENIX-5250 (This issue from our
production environments, we had used HBase 2.0.2 in Phoenix 5.0).

2.  We should have a clear JIRA list for a 5.0.1, I am not sure what JIRAs
should go into it, only critical and blocked JIRAs apply for a 5.0.1?  If I
know what priority need to pick, I will collect them.



Jaanai Zhang
Best regards!



Thomas D'Silva  于2019年4月30日周二 上午10:06写道：


Jaanai,

We are waiting for a few omid bug fixes to do the 4.15/5.1 release that
will have splittable system catalog and the omid integration.
Are you interested in doing a 5.0.1 release that has the HBase 2.0.x
compatibility fixes that were discussed in a previous thread[1]?

The steps to create a RC are straightforward and documented here:
https://phoenix.apache.org/release.html.
The main thing you need to do is to add your code signing key to
https://dist.apache.org/repos/dist/dev/phoenix/KEYS (follow the steps at
the start of that file)
and then commit using svn. Then you can follow the rest of the steps listed
in "How to do a release"

Thanks,
Thomas

[1]

https://lists.apache.org/thread.html/99fcc737d7a8f82ddffb1b34a64f7099f7909900b8bea36dd6afca16@%3Cdev.phoenix.apache.org%3E

On Mon, Apr 29, 2019 at 6:33 PM Jaanai Zhang 
wrote:


I would like to volunteer for a new 5.x release if someone can guide me
release process. Thanks.


Jaanai Zhang
Best regards!



Josh Elser  于2019年4月30日周二 上午12:39写道：


Hiya folks,

It's about that time for another board report. Please reply here with
anything of merit that you think the board might find
interesting/useful. As a reminder, they board is typically more
concerned with high-level project/community details than the
nuts-and-bolts of the code changes for the project.

On my radar already is...

* Multiple new committers and PMC'ers (thanks so much to the folks who
have been driving votes!)
* NoSQL day in May
* 4.14.2 in vote
* Need for a new 5.x.y release (if there are no volunteers, I may have
to find the time to make this happen. It's been too long)

Thanks!

- Josh

Re: [ANNOUNCE] New Phoenix committer: Mihir Monani

[cheng...@apache.org]

Congratulations and welcome Mihir!







在 2019-04-30 14:35:54，"Jaanai Zhang"  写道：
>congrats! Mihir.
>
>
>   Jaanai Zhang
>   Best regards!
>
>
>
>Mihir Monani  于2019年4月30日周二 下午1:13写道：
>
>> Thanks Thomas and Apache Phoenix Community. :)
>>
>> On Sun, Apr 28, 2019 at 6:12 AM Thomas D'Silva  wrote:
>>
>> > On behalf of the Apache Phoenix PMC, I am pleased to announce that
>> > Mihir Monani has accepted our invitation to become a committer.
>> > Mihir has done some nice work fixing several bugs related to indexing[1].
>> >
>> > Please welcome him to the Apache Phoenix team.
>> >
>> > Thanks,
>> > Thomas
>> >
>> > [1]
>> >
>> >
>> https://issues.apache.org/jira/browse/PHOENIX-5199?jql=project%20%3D%20PHOENIX%20AND%20assignee%3D%22mihir6692%22%20AND%20status%3DResolved
>> >
>>
>>
>> --
>> Mihir Monani
>> (+91)-9429473434
>>

Re: [ANNOUNCE] New Phoenix committer: Mihir Monani

[Jaanai Zhang]

congrats! Mihir.


   Jaanai Zhang
   Best regards!



Mihir Monani  于2019年4月30日周二 下午1:13写道：

> Thanks Thomas and Apache Phoenix Community. :)
>
> On Sun, Apr 28, 2019 at 6:12 AM Thomas D'Silva  wrote:
>
> > On behalf of the Apache Phoenix PMC, I am pleased to announce that
> > Mihir Monani has accepted our invitation to become a committer.
> > Mihir has done some nice work fixing several bugs related to indexing[1].
> >
> > Please welcome him to the Apache Phoenix team.
> >
> > Thanks,
> > Thomas
> >
> > [1]
> >
> >
> https://issues.apache.org/jira/browse/PHOENIX-5199?jql=project%20%3D%20PHOENIX%20AND%20assignee%3D%22mihir6692%22%20AND%20status%3DResolved
> >
>
>
> --
> Mihir Monani
> (+91)-9429473434
>

Re: Board report due in ~1 week

[Jaanai Zhang]

I will try doing release 5.0.1. Tow things need to confirm:

1.  According to the thread[1],  a new Phoenix 5.0.1 will support HBase
2.0.2, right? As far as I know, some improvements of HBase 2.0.2 will cause
some critical issues, for example,
https://issues.apache.org/jira/browse/PHOENIX-5250 (This issue from our
production environments, we had used HBase 2.0.2 in Phoenix 5.0).

2.  We should have a clear JIRA list for a 5.0.1, I am not sure what JIRAs
should go into it, only critical and blocked JIRAs apply for a 5.0.1?  If I
know what priority need to pick, I will collect them.



   Jaanai Zhang
   Best regards!



Thomas D'Silva  于2019年4月30日周二 上午10:06写道：

> Jaanai,
>
> We are waiting for a few omid bug fixes to do the 4.15/5.1 release that
> will have splittable system catalog and the omid integration.
> Are you interested in doing a 5.0.1 release that has the HBase 2.0.x
> compatibility fixes that were discussed in a previous thread[1]?
>
> The steps to create a RC are straightforward and documented here:
> https://phoenix.apache.org/release.html.
> The main thing you need to do is to add your code signing key to
> https://dist.apache.org/repos/dist/dev/phoenix/KEYS (follow the steps at
> the start of that file)
> and then commit using svn. Then you can follow the rest of the steps listed
> in "How to do a release"
>
> Thanks,
> Thomas
>
> [1]
>
> https://lists.apache.org/thread.html/99fcc737d7a8f82ddffb1b34a64f7099f7909900b8bea36dd6afca16@%3Cdev.phoenix.apache.org%3E
>
> On Mon, Apr 29, 2019 at 6:33 PM Jaanai Zhang 
> wrote:
>
> > I would like to volunteer for a new 5.x release if someone can guide me
> > release process. Thanks.
> >
> > 
> >Jaanai Zhang
> >Best regards!
> >
> >
> >
> > Josh Elser  于2019年4月30日周二 上午12:39写道：
> >
> > > Hiya folks,
> > >
> > > It's about that time for another board report. Please reply here with
> > > anything of merit that you think the board might find
> > > interesting/useful. As a reminder, they board is typically more
> > > concerned with high-level project/community details than the
> > > nuts-and-bolts of the code changes for the project.
> > >
> > > On my radar already is...
> > >
> > > * Multiple new committers and PMC'ers (thanks so much to the folks who
> > > have been driving votes!)
> > > * NoSQL day in May
> > > * 4.14.2 in vote
> > > * Need for a new 5.x.y release (if there are no volunteers, I may have
> > > to find the time to make this happen. It's been too long)
> > >
> > > Thanks!
> > >
> > > - Josh
> > >
> >
>

[jira] [Updated] (PHOENIX-5250) The accumulated wal files can't be cleaned

[jaanai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-5250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jaanai updated PHOENIX-5250:

Affects Version/s: (was: 5.0.0)

> The accumulated wal files can't be cleaned
> --
>
> Key: PHOENIX-5250
> URL: https://issues.apache.org/jira/browse/PHOENIX-5250
> Project: Phoenix
>  Issue Type: Bug
>Reporter: jaanai
>Priority: Blocker
> Attachments: image-2019-04-19-21-31-27-888.png
>
>
> Because of the modification of HBASE-20781,  the faked WALEdits won't be 
> filtered, all WALEdits will be saved into Memstore with a status that 
> inMemstore is true(uses WAL->append method).
> !image-2019-04-19-21-31-27-888.png|width=755,height=310!
> The family array of IndexedKeyValue is WALEdit.METAFAMILY that is used to 
> describe a fake WALEdit, and it will put into Memstore with WALedits of data 
> table during sync global index.
> WAL files can't be cleaned, expect for resarting RS, Many WAL files will 
> decrease the percent of disk free.  
> !https://gw.alicdn.com/tfscom/TB1n3cDQVzqK1RjSZFCXXbbxVXa.png|width=422,height=159!
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)