[jira] [Resolved] (PHOENIX-3398) ROW_TIMESTAMP set in CONSTRAINT field does not set isRowTimestamp in ColumnDef
[
https://issues.apache.org/jira/browse/PHOENIX-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor resolved PHOENIX-3398.
---
Resolution: Invalid
This is working as designed, as the isRowTimestamp is stored in the
PrimaryKeyConstraint if defined in the primary key constraint and recognized
during the create table process in MetaDataClient.
> ROW_TIMESTAMP set in CONSTRAINT field does not set isRowTimestamp in ColumnDef
> --
>
> Key: PHOENIX-3398
> URL: https://issues.apache.org/jira/browse/PHOENIX-3398
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Kevin Liew
>
> The following DDL will result in `isRowTimestamp` being set in ColumnDef.
> {code:java}
> String ddl = "CREATE TABLE IF NOT EXISTS " + table + " ("
> + "pk BIGINT DEFAULT 5 PRIMARY KEY ROW_TIMESTAMP)";
> {code}
> The following DDL will not:
> {code:java}
> String ddl = "CREATE TABLE IF NOT EXISTS " + table + " ("
> + "pk1 INTEGER NOT NULL,"
> + "pk2 BIGINT NOT NULL DEFAULT 5,"
> + "CONSTRAINT NAME_PK PRIMARY KEY (pk1, pk2 ROW_TIMESTAMP))";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598934#comment-15598934 ] Kevin Liew commented on PHOENIX-476: I filed PHOENIX-3397 for that. Also, PHOENIX-3398 regarding `isRowTimestamp` not being set when we set ROW_TIMESTAMP in the CONSTRAINTS field, which should be possible according to the website. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3398) ROW_TIMESTAMP set in CONSTRAINT field does not set isRowTimestamp in ColumnDef
Kevin Liew created PHOENIX-3398:
---
Summary: ROW_TIMESTAMP set in CONSTRAINT field does not set
isRowTimestamp in ColumnDef
Key: PHOENIX-3398
URL: https://issues.apache.org/jira/browse/PHOENIX-3398
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Reporter: Kevin Liew
The following DDL will result in `isRowTimestamp` being set in ColumnDef.
{code:java}
String ddl = "CREATE TABLE IF NOT EXISTS " + table + " ("
+ "pk BIGINT DEFAULT 5 PRIMARY KEY ROW_TIMESTAMP)";
{code}
The following DDL will not:
{code:java}
String ddl = "CREATE TABLE IF NOT EXISTS " + table + " ("
+ "pk1 INTEGER NOT NULL,"
+ "pk2 BIGINT NOT NULL DEFAULT 5,"
+ "CONSTRAINT NAME_PK PRIMARY KEY (pk1, pk2 ROW_TIMESTAMP))";
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (PHOENIX-3397) Compiled ARRAY date-time literals cannot be parsed as literals
Kevin Liew created PHOENIX-3397:
---
Summary: Compiled ARRAY date-time literals cannot be parsed as
literals
Key: PHOENIX-3397
URL: https://issues.apache.org/jira/browse/PHOENIX-3397
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Reporter: Kevin Liew
Priority: Minor
We ran into an issue in PHOENIX-476 where: if you compile an ARRAY expression
with date-time literals, store the resulting expression string in the
metastore, and later try to parse it as a literal, you will get an error:
{code:java}
org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type
mismatch. ERROR 203 (22005): Type mismatch. ERROR 203 (22005): Type mismatch.
ERROR 203 (22005): Type mismatch. VARCHAR cannot be coerced to TIME
at
org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:74)
at
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
at
org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:129)
at
org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:118)
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:107)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:751)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:695)
at
org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at
org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at
org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at
org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:778)
at
org.apache.phoenix.end2end.DefaultColumnValueIT.testDefaultArrays(DefaultColumnValueIT.java:732)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
{code}
This is due to the following code in ExpressionCompiler.
{code:java}
public Expression visitLeave(ArrayConstructorNode node, List
children) throws SQLException {
...
ArrayConstructorExpression arrayExpression = new
ArrayConstructorExpression(children, arrayElemDataType,
[jira] [Commented] (PHOENIX-3390) Custom UDAF for HyperLogLogPlus
[ https://issues.apache.org/jira/browse/PHOENIX-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598761#comment-15598761 ] Swapna Kasula commented on PHOENIX-3390: The function APPROX_SUM is very similar to the existing aggregate function we have is 'SUM(X)', to allow in the select statements. However we pass col to the SUM(X), APPROX_SUM works the same way as : select APPROX_SUM(col) from table; This is how I visualize: Example: rowkeycol1 col2 google hll1 hll2 yahoo hll3 hll4 msn hll5 hll6 select APPROX_SUM(col1) from table; //returns union of (hll1 U hll3 U hll5) for all rows select APPROX_SUM(col1) from table where rowkey ='google' or rowkey ='yahoo' // this returns unions of (hll1 U hll3) > Custom UDAF for HyperLogLogPlus > --- > > Key: PHOENIX-3390 > URL: https://issues.apache.org/jira/browse/PHOENIX-3390 > Project: Phoenix > Issue Type: New Feature >Reporter: Swapna Kasula >Priority: Minor > > With ref # PHOENIX-2069 > Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a > Hyperloglog. > select hllUnion(col1) from table; //returns a Hyperloglog, which is the > union of all hyperloglog's from all rows for column 'col1' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3390) Custom UDAF for HyperLogLogPlus
[
https://issues.apache.org/jira/browse/PHOENIX-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swapna Kasula updated PHOENIX-3390:
---
Description:
With ref # PHOENIX-2069
Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
Hyperloglog.
select hllUnion(col1) from table; //returns a Hyperloglog, which is the union
of all hyperloglog's from all rows for column 'col1'
was:
With ref # PHOENIX-2069
Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
Hyperloglog.
select hllUnion("col1") from table; //returns a Hyperloglog, which is the
union of all hyperloglog's from all rows for column 'col1'
> Custom UDAF for HyperLogLogPlus
> ---
>
> Key: PHOENIX-3390
> URL: https://issues.apache.org/jira/browse/PHOENIX-3390
> Project: Phoenix
> Issue Type: New Feature
>Reporter: Swapna Kasula
>Priority: Minor
>
> With ref # PHOENIX-2069
> Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
> Hyperloglog.
> select hllUnion(col1) from table; //returns a Hyperloglog, which is the
> union of all hyperloglog's from all rows for column 'col1'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598665#comment-15598665 ] James Taylor commented on PHOENIX-476: -- Still sounds like a bug lurking here, but it doesn't need to be fixed for this JIRA. But let's file one with the above info so we don't lose track of it. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598651#comment-15598651
]
Kevin Liew edited comment on PHOENIX-476 at 10/22/16 11:08 PM:
---
I got it working. With the recent changes, we access the context again during
the UPSERT so we can compile arbitrary isConstant expressions. I just needed to
revert the older changes where we were compiling the expression to a literal in
CreateTableCompiler and storing the expression string obtained from that.
For scalar date-time, the expression was untouched. For arrays, it was being
modified below:
{code:java}
public Expression visitLeave(ArrayConstructorNode node, List
children) throws SQLException {
...
ArrayConstructorExpression arrayExpression = new
ArrayConstructorExpression(children, arrayElemDataType, rowKeyOrderOptimizable);
if (ExpressionUtil.isConstant(arrayExpression)) {
for (int i = 0; i < children.size(); i++) {
Expression child = children.get(i);
child.evaluate(null, ptr);
Object value = null;
if (child.getDataType() == null) {
value = arrayElemDataType.toObject(ptr,
theArrayElemDataType, child.getSortOrder());
} else {
value = arrayElemDataType.toObject(ptr,
child.getDataType(), child.getSortOrder());
}
elements[i] = LiteralExpression.newConstant(value,
theArrayElemDataType, child.getDeterminism()).getValue();
}
Object value =
PArrayDataType.instantiatePhoenixArray(arrayElemDataType, elements);
return LiteralExpression.newConstant(value,
PDataType.fromTypeId(arrayElemDataType.getSqlType() +
PDataType.ARRAY_TYPE_BASE), null, null, arrayExpression.getSortOrder(),
Determinism.ALWAYS, rowKeyOrderOptimizable);
}
{code}
It looks that behavior was intended for some other use case but the issue might
show up again.
Its just testing and cleanup left now.
was (Author: kliew):
I got it working. With the recent changes, we access the context again during
the UPSERT so we can compile arbitrary isConstant expressions. I just needed to
revert the older changes where we were compiling the expression to a literal in
CreateTableCompiler and storing the expression string obtained from that.
For scalar date-time, the expression was untouched. For arrays, it was being
modified below:
{code:java}
public Expression visitLeave(ArrayConstructorNode node, List
children) throws SQLException {
...
ArrayConstructorExpression arrayExpression = new
ArrayConstructorExpression(children, arrayElemDataType, rowKeyOrderOptimizable);
if (ExpressionUtil.isConstant(arrayExpression)) {
for (int i = 0; i < children.size(); i++) {
Expression child = children.get(i);
child.evaluate(null, ptr);
Object value = null;
if (child.getDataType() == null) {
value = arrayElemDataType.toObject(ptr,
theArrayElemDataType, child.getSortOrder());
} else {
value = arrayElemDataType.toObject(ptr,
child.getDataType(), child.getSortOrder());
}
elements[i] = LiteralExpression.newConstant(value,
theArrayElemDataType, child.getDeterminism()).getValue();
}
Object value =
PArrayDataType.instantiatePhoenixArray(arrayElemDataType, elements);
return LiteralExpression.newConstant(value,
PDataType.fromTypeId(arrayElemDataType.getSqlType() +
PDataType.ARRAY_TYPE_BASE), null, null, arrayExpression.getSortOrder(),
Determinism.ALWAYS, rowKeyOrderOptimizable);
}
{code}
Its just testing and cleanup left now.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598651#comment-15598651
]
Kevin Liew commented on PHOENIX-476:
I got it working. With the recent changes, we access the context again during
the UPSERT so we can compile arbitrary isConstant expressions. I just needed to
revert the older changes where we were compiling the expression to a literal in
CreateTableCompiler and storing the expression string obtained from that.
For scalar date-time, the expression was untouched. For arrays, it was being
modified below:
{code:java}
public Expression visitLeave(ArrayConstructorNode node, List
children) throws SQLException {
...
ArrayConstructorExpression arrayExpression = new
ArrayConstructorExpression(children, arrayElemDataType, rowKeyOrderOptimizable);
if (ExpressionUtil.isConstant(arrayExpression)) {
for (int i = 0; i < children.size(); i++) {
Expression child = children.get(i);
child.evaluate(null, ptr);
Object value = null;
if (child.getDataType() == null) {
value = arrayElemDataType.toObject(ptr,
theArrayElemDataType, child.getSortOrder());
} else {
value = arrayElemDataType.toObject(ptr,
child.getDataType(), child.getSortOrder());
}
elements[i] = LiteralExpression.newConstant(value,
theArrayElemDataType, child.getDeterminism()).getValue();
}
Object value =
PArrayDataType.instantiatePhoenixArray(arrayElemDataType, elements);
return LiteralExpression.newConstant(value,
PDataType.fromTypeId(arrayElemDataType.getSqlType() +
PDataType.ARRAY_TYPE_BASE), null, null, arrayExpression.getSortOrder(),
Determinism.ALWAYS, rowKeyOrderOptimizable);
}
{code}
Its just testing and cleanup left now.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3390) Custom UDAF for HyperLogLogPlus
[
https://issues.apache.org/jira/browse/PHOENIX-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598645#comment-15598645
]
James Taylor commented on PHOENIX-3390:
---
The way I think this could be more generally exposed is through a new set of
approximate aggregate functions like APPROX_COUNT, APPROX_SUM, etc. You
wouldn't want to quote column names either - that wouldn't be very natural and
you'd lose the type checking ability that's important in SQL.
> Custom UDAF for HyperLogLogPlus
> ---
>
> Key: PHOENIX-3390
> URL: https://issues.apache.org/jira/browse/PHOENIX-3390
> Project: Phoenix
> Issue Type: New Feature
>Reporter: Swapna Kasula
>Priority: Minor
>
> With ref # PHOENIX-2069
> Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
> Hyperloglog.
> select hllUnion("col1") from table; //returns a Hyperloglog, which is the
> union of all hyperloglog's from all rows for column 'col1'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598637#comment-15598637 ] James Taylor commented on PHOENIX-476: -- Good find. Please file a separate JIRA for that and don't worry about it for now (i.e. Don't use date/time types in arrays for your tests). > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598382#comment-15598382
]
Kevin Liew edited comment on PHOENIX-476 at 10/22/16 7:52 PM:
--
We compile the string to an expression in CreateTableCompiler and save that to
the metastore. For arrays: the date-time is represented as a string (without
the type specifier) in the new expression string.
When we call `parseExpression` while doing UPSERT, it only sees the string
without the type information.
{code:java}
ExpressionCompiler compiler = new ExpressionCompiler(context);
ParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseExpression();
Expression defaultExpression = defaultParseNode.accept(compiler);
{code}
For scalar columns, the type specifier is not removed after compiling an
Expression (ie. "DATE '1900-10-01 14:03:22.559'").
I don't think it is expected for the type specifier to be removed after
compiling array values.
was (Author: kliew):
We compile the string to an expression contained in a ColumnDef. For arrays:
the date-time is represented as a string (without the type specifier) in the
new expression string.
When we call `parseExpression`, it only sees the string without the type
information.
{code:java}
ExpressionCompiler compiler = new ExpressionCompiler(context);
ParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseExpression();
Expression defaultExpression = defaultParseNode.accept(compiler);
{code}
For scalar columns, the type specifier is not removed after compiling an
Expression (ie. "DATE '1900-10-01 14:03:22.559'").
I don't think it is expected for the type specifier to be removed after
compiling array values.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598382#comment-15598382
]
Kevin Liew commented on PHOENIX-476:
We compile the string to an expression contained in a ColumnDef. For arrays:
the date-time is represented as a string (without the type specifier) in the
new expression string.
When we call `parseExpression`, it only sees the string without the type
information.
{code:java}
ExpressionCompiler compiler = new ExpressionCompiler(context);
ParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseExpression();
Expression defaultExpression = defaultParseNode.accept(compiler);
{code}
For scalar columns, the type specifier is not removed after compiling an
Expression (ie. "DATE '1900-10-01 14:03:22.559'").
I don't think it is expected for the type specifier to be removed after
compiling array values.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3390) Custom UDAF for HyperLogLogPlus
[
https://issues.apache.org/jira/browse/PHOENIX-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swapna Kasula updated PHOENIX-3390:
---
Description:
With ref # PHOENIX-2069
Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
Hyperloglog.
select hllUnion("col1") from table; //returns a Hyperloglog, which is the
union of all hyperloglog's from all rows for column 'col1'
was:
With ref # PHOENIX-2069
Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
Hyperloglog.
> Custom UDAF for HyperLogLogPlus
> ---
>
> Key: PHOENIX-3390
> URL: https://issues.apache.org/jira/browse/PHOENIX-3390
> Project: Phoenix
> Issue Type: New Feature
>Reporter: Swapna Kasula
>Priority: Minor
>
> With ref # PHOENIX-2069
> Custome UDAF to aggregate/union of Hyperloglog's of a column and returns a
> Hyperloglog.
> select hllUnion("col1") from table; //returns a Hyperloglog, which is the
> union of all hyperloglog's from all rows for column 'col1'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598342#comment-15598342 ] James Taylor commented on PHOENIX-476: -- Looks correct - you can verify by using a SELECT statement with the same literal expression. You're still using parseExpression(), right? Might be a bug. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598326#comment-15598326
]
Kevin Liew commented on PHOENIX-476:
The byte coercion fix for data types also fixed byte coercion for arrays.
Right now I'm running into a problem with default values for date-time types.
"time '1900-10-01 14:03:22.559'" worked for non array types.
I tried a few different variations of:
{code:sql}
tim TIME[5] DEFAULT ARRAY[time '1900-10-01 14:03:22.559', time '1901-10-01
14:03:22.559']
{code}
{code}
org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type
mismatch. ERROR 203 (22005): Type mismatch. ERROR 203 (22005): Type mismatch.
ERROR 203 (22005): Type mismatch. VARCHAR cannot be coerced to TIME
at
org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:74)
at
org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
at
org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:129)
at
org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:118)
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:107)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:751)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:695)
at
org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at
org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at
org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at
org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:778)
at
org.apache.phoenix.end2end.DefaultColumnValueIT.testDefaultArrays(DefaultColumnValueIT.java:732)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
at
com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
{code}
I'll try alternate syntax, but was the above syntactically correct?
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598316#comment-15598316
]
Hadoop QA commented on PHOENIX-476:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12834822/PHOENIX-476.8.patch
against master branch at commit 8f5c8cfbd1c1f7894e4cc88e2257d245d1d8c3bf.
ATTACHMENT ID: 12834822
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
43 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ResultSet rs = conn.createStatement().executeQuery("SELECT " +
projection + " FROM T_DEFAULT WHERE pk1 = 1");
+rs = conn.createStatement().executeQuery("SELECT " + projection + "
FROM T_DEFAULT WHERE pk1 = 11");
+ResultSet rs = conn.createStatement().executeQuery("SELECT " +
projection + " FROM T_DEFAULT WHERE pk1 = 1");
+rs = conn.createStatement().executeQuery("SELECT " + projection + "
FROM T_DEFAULT WHERE pk1 = 11");
+rs = conn.createStatement().executeQuery("SELECT " + projection + "
FROM T_DEFAULT WHERE pk1 = 1");
+rs = conn.createStatement().executeQuery("SELECT " + projection + "
FROM T_DEFAULT WHERE pk1 = 11");
+ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
T_DEFAULT2 WHERE pk1 = 1");
+ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
T_DEFAULT2 WHERE pk1 = 1");
+ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
table + " WHERE pk = 1");
+ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM " +
table + " WHERE pk = 1");
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DefaultColumnValueIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CreateSchemaIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/633//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/633//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/633//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/633//console
This message is automatically generated.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598313#comment-15598313 ] James Taylor commented on PHOENIX-476: -- Is anything special required for array support? > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598280#comment-15598280 ] Kevin Liew edited comment on PHOENIX-476 at 10/22/16 6:36 PM: -- Thanks James, that works. I'll get started on array support and further revision and testing . was (Author: kliew): Thanks James, that works. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598280#comment-15598280 ] Kevin Liew commented on PHOENIX-476: Thanks James, that works. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598264#comment-15598264
]
James Taylor commented on PHOENIX-476:
--
Scratch that - it's not going to work to have a null PhoenixConnection. Do this
instead inside of PTableImpl:
{code}
String url = PhoenixRuntime.JDBC_PROTOCOL +
PhoenixRuntime.JDBC_PROTOCOL_SEPARATOR + PhoenixRuntime.CONNECTIONLESS
PhoenixConnection conn =
DriverManager.getConnection(url).unwrap(PhoenixConnection.class);
StatementContext context = new StatementContext(new PhoenixStatement(conn));
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598256#comment-15598256
]
James Taylor commented on PHOENIX-476:
--
I see - yes that'll still be a problem. Try this to create a StatementContext
within the PTableImpl.newKey() instead of passing it in:
{code}
StatementContext context = new StatementContext(new PhoenixStatement(null));
{code}
I think you might be able to get away with having a null Connection when you're
only parsing an expression that is a literal. If that works, you can still with
your last patch, but without passing through a StatementContext to the
PTableImpl method.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598254#comment-15598254
]
Hadoop QA commented on PHOENIX-476:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12834821/PHOENIX-476.8.patch
against master branch at commit 8f5c8cfbd1c1f7894e4cc88e2257d245d1d8c3bf.
ATTACHMENT ID: 12834821
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 69 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/632//console
This message is automatically generated.
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598242#comment-15598242 ] Kevin Liew edited comment on PHOENIX-476 at 10/22/16 6:10 PM: -- Oops, sorry I had to reset my environment so the patch is messy due to IDE changes. I cleaned up and reattached the patch. was (Author: kliew): Oops, sorry I had to reset my environment so the patch is messy due to IDE changes. I will clean up the patch and attach again. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated PHOENIX-476: --- Attachment: (was: PHOENIX-476.8.patch) > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated PHOENIX-476: --- Attachment: PHOENIX-476.8.patch > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598242#comment-15598242 ] Kevin Liew commented on PHOENIX-476: Oops, sorry I had to reset my environment so the patch is messy due to IDE changes. I will clean up the patch and attach again. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated PHOENIX-476: --- Attachment: PHOENIX-476.8.patch I attached what I have so far. I debugged to ensure that `SQLParse.literal` is called on the value `-100` as a literal, not expanded as an expression. The issue seems to be in the parser. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.8.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598232#comment-15598232
]
James Taylor commented on PHOENIX-476:
--
So for this to work, in CreateTableCompiler, after you evaluate the DEFAULT
expression, you'll need to replace the existing ColumnDef with a new one that
you'll instantiate using the same values except for the expressionStr which
you'll replace with the evaluatedDefaultExpression.toString() value.
{code}
ColumnDef(ColumnName columnDefName, String sqlTypeName, boolean isArray,
Integer arrSize, Boolean isNull, Integer maxLength,
Integer scale, boolean isPK, SortOrder sortOrder, String
expressionStr, boolean isRowTimestamp) {
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598230#comment-15598230 ] Kevin Liew commented on PHOENIX-476: That's what I did, but the literal `-100` fails to parse as a literal. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598225#comment-15598225
]
James Taylor commented on PHOENIX-476:
--
No parser changes are required:
-In CreateTableCompiler, when you compile the defaultExpression, you'll end up
with a LiteralExpression (i.e. DEFAULT -2 + 7 would be turned into a
LiteralExpression of 5). Create a new CreateTableStatement copy constructor
that passes in a createTableStatement and an expressionStr. Use this to create
a new CreateTableStatement using defaultExpression.toString() to form the
literal expression string which you'll pass through to the MetaDataClient to
guarantee that the default expression is always a literal.
- By evaluating the default expression at CREATE TABLE time, and storing the
evaluated expression string value, you'll know you have literal. When reading
and parsing the default value back in PTableImpl, you can assume it's already a
literal and parse it as you were doing in your previous patch:
{code}
+LiteralParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseLiteral();
+LiteralExpression defaultLiteral =
LiteralExpression.newConstant(defaultParseNode.getValue(),
column.getDataType(), column.getMaxLength(), column.getScale(),
column.getSortOrder(), Determinism.ALWAYS);
+ImmutableBytesWritable valuePtr = new
ImmutableBytesWritable();
+defaultLiteral.evaluate(null, valuePtr);
+byteValue =
ByteUtil.copyKeyBytesIfNecessary(valuePtr);
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598225#comment-15598225
]
James Taylor edited comment on PHOENIX-476 at 10/22/16 5:53 PM:
No parser changes are required:
- In CreateTableCompiler, when you compile the defaultExpression, you'll end up
with a LiteralExpression (i.e. DEFAULT -2 + 7 would be turned into a
LiteralExpression of 5). Create a new CreateTableStatement copy constructor
that passes in a createTableStatement and an expressionStr. Use this to create
a new CreateTableStatement using defaultExpression.toString() to form the
literal expression string which you'll pass through to the MetaDataClient to
guarantee that the default expression is always a literal.
- By evaluating the default expression at CREATE TABLE time, and storing the
evaluated expression string value, you'll know you have literal. When reading
and parsing the default value back in PTableImpl, you can assume it's already a
literal and parse it as you were doing in your previous patch:
{code}
+LiteralParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseLiteral();
+LiteralExpression defaultLiteral =
LiteralExpression.newConstant(defaultParseNode.getValue(),
column.getDataType(), column.getMaxLength(), column.getScale(),
column.getSortOrder(), Determinism.ALWAYS);
+ImmutableBytesWritable valuePtr = new
ImmutableBytesWritable();
+defaultLiteral.evaluate(null, valuePtr);
+byteValue =
ByteUtil.copyKeyBytesIfNecessary(valuePtr);
{code}
was (Author: jamestaylor):
No parser changes are required:
-In CreateTableCompiler, when you compile the defaultExpression, you'll end up
with a LiteralExpression (i.e. DEFAULT -2 + 7 would be turned into a
LiteralExpression of 5). Create a new CreateTableStatement copy constructor
that passes in a createTableStatement and an expressionStr. Use this to create
a new CreateTableStatement using defaultExpression.toString() to form the
literal expression string which you'll pass through to the MetaDataClient to
guarantee that the default expression is always a literal.
- By evaluating the default expression at CREATE TABLE time, and storing the
evaluated expression string value, you'll know you have literal. When reading
and parsing the default value back in PTableImpl, you can assume it's already a
literal and parse it as you were doing in your previous patch:
{code}
+LiteralParseNode defaultParseNode = new
SQLParser(column.getExpressionStr()).parseLiteral();
+LiteralExpression defaultLiteral =
LiteralExpression.newConstant(defaultParseNode.getValue(),
column.getDataType(), column.getMaxLength(), column.getScale(),
column.getSortOrder(), Determinism.ALWAYS);
+ImmutableBytesWritable valuePtr = new
ImmutableBytesWritable();
+defaultLiteral.evaluate(null, valuePtr);
+byteValue =
ByteUtil.copyKeyBytesIfNecessary(valuePtr);
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15597332#comment-15597332
]
Kevin Liew edited comment on PHOENIX-476 at 10/22/16 5:30 PM:
--
Hi [~jamestaylor], I made the suggested changes to recreate the create-table
statement with the literal value string that we get after compiling the
statement.
Without a StatementContext, that takes us back to the issue with parsing
negative literal numbers at `new
SQLParser(column.getExpressionStr()).parseLiteral()`.
I tried defining a new parse function without adding new tokens so that other
functions won't be affected but the parser throws an exception during the
prediction stage.
{code}
// Get a string, integer, double, date, boolean, or NULL value.
// Able to parse negative numbers.
any_literal returns [LiteralParseNode ret]
: s=STRING_LITERAL {
ret = factory.literal(s.getText());
}
| n=NUMBER {
ret = factory.wholeNumber(n.getText());
}
| nn=MINUS DIGIT+ {
ret = factory.wholeNumber(nn.getText());
}
| d=DECIMAL {
ret = factory.realNumber(d.getText());
}
| nd=MINUS DIGIT+ DOT POSINTEGER {
ret = factory.realNumber(nd.getText());
}
| dbl=DOUBLE {
ret = factory.literal(Double.valueOf(dbl.getText()));
}
| ndbl='.' POSINTEGER Exponent | MINUS DIGIT+ DOT Exponent | MINUS DIGIT+
(DOT (POSINTEGER (Exponent)?)? | Exponent) {
ret = factory.literal(Double.valueOf(ndbl.getText()));
}
| NULL {ret = factory.literal(null);}
| TRUE {ret = factory.literal(Boolean.TRUE);}
| FALSE {ret = factory.literal(Boolean.FALSE);}
| dt=identifier t=STRING_LITERAL {
try {
ret = factory.literal(t.getText(), dt);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
;
{code}
was (Author: kliew):
Hi [~jamestaylor], I made the suggested changes to recreate the create-table
statement with the literal value string that we get after compiling the
statement.
Without a StatementContext, that takes us back to the issue with parsing
negative literals at `new SQLParser(column.getExpressionStr()).parseLiteral()`.
I tried defining a new parse function without adding new tokens so that other
functions won't be affected but the parser throws an exception during the
prediction stage.
{code}
// Get a string, integer, double, date, boolean, or NULL value.
// Able to parse negative numbers.
any_literal returns [LiteralParseNode ret]
: s=STRING_LITERAL {
ret = factory.literal(s.getText());
}
| n=NUMBER {
ret = factory.wholeNumber(n.getText());
}
| nn=MINUS DIGIT+ {
ret = factory.wholeNumber(nn.getText());
}
| d=DECIMAL {
ret = factory.realNumber(d.getText());
}
| nd=MINUS DIGIT+ DOT POSINTEGER {
ret = factory.realNumber(nd.getText());
}
| dbl=DOUBLE {
ret = factory.literal(Double.valueOf(dbl.getText()));
}
| ndbl='.' POSINTEGER Exponent | MINUS DIGIT+ DOT Exponent | MINUS DIGIT+
(DOT (POSINTEGER (Exponent)?)? | Exponent) {
ret = factory.literal(Double.valueOf(ndbl.getText()));
}
| NULL {ret = factory.literal(null);}
| TRUE {ret = factory.literal(Boolean.TRUE);}
| FALSE {ret = factory.literal(Boolean.FALSE);}
| dt=identifier t=STRING_LITERAL {
try {
ret = factory.literal(t.getText(), dt);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
;
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[ https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15598196#comment-15598196 ] Kevin Liew commented on PHOENIX-476: One possible solution is to create a second parser class specifically for literals, which would give us the freedom to define any tokens without affecting expression parsing. > Support declaration of DEFAULT in CREATE statement > -- > > Key: PHOENIX-476 > URL: https://issues.apache.org/jira/browse/PHOENIX-476 > Project: Phoenix > Issue Type: Task >Affects Versions: 3.0-Release >Reporter: James Taylor >Assignee: Kevin Liew > Labels: enhancement > Fix For: 4.9.0 > > Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch, > PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch, > PHOENIX-476.7.patch, PHOENIX-476.patch > > > Support the declaration of a default value in the CREATE TABLE/VIEW statement > like this: > CREATE TABLE Persons ( > Pid int NOT NULL PRIMARY KEY, > LastName varchar(255) NOT NULL, > FirstName varchar(255), > Address varchar(255), > City varchar(255) DEFAULT 'Sandnes' > ) > To implement this, we'd need to: > 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through > the value when the table is created (in MetaDataClient). > 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value > is present, since the column will never be null. > 3. add a getDefaultValue() accessor in PColumn > 4. for a row key column, during UPSERT use the default value if no value was > specified for that column. This could be done in the PTableImpl.newKey method. > 5. for a key value column with a default value, we can get away without > incurring any storage cost. Although a little bit of extra effort than if we > persisted the default value on an UPSERT for key value columns, this approach > has the benefit of not incurring any storage cost for a default value. > * serialize any default value into KeyValueColumnExpression > * in the evaluate method of KeyValueColumnExpression, conditionally use > the default value if the column value is not present. If doing partial > evaluation, you should not yet return the default value, as we may not have > encountered the the KeyValue for the column yet (since a filter evaluates > each time it sees each KeyValue, and there may be more than one KeyValue > referenced in the expression). Partial evaluation is determined by calling > Tuple.isImmutable(), where false means it is NOT doing partial evaluation, > while true means it is. > * modify EvaluateOnCompletionVisitor by adding a visitor method for > RowKeyColumnExpression and KeyValueColumnExpression to set > evaluateOnCompletion to true if they have a default value specified. This > will cause filter evaluation to execute one final time after all KeyValues > for a row have been seen, since it's at this time we know we should use the > default value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15597332#comment-15597332
]
Kevin Liew edited comment on PHOENIX-476 at 10/22/16 5:17 PM:
--
Hi [~jamestaylor], I made the suggested changes to recreate the create-table
statement with the literal value string that we get after compiling the
statement.
Without a StatementContext, that takes us back to the issue with parsing
negative literals at `new SQLParser(column.getExpressionStr()).parseLiteral()`.
I tried defining a new parse function without adding new tokens so that other
functions won't be affected but the parser throws an exception during the
prediction stage.
{code}
// Get a string, integer, double, date, boolean, or NULL value.
// Able to parse negative numbers.
any_literal returns [LiteralParseNode ret]
: s=STRING_LITERAL {
ret = factory.literal(s.getText());
}
| n=NUMBER {
ret = factory.wholeNumber(n.getText());
}
| nn=MINUS DIGIT+ {
ret = factory.wholeNumber(nn.getText());
}
| d=DECIMAL {
ret = factory.realNumber(d.getText());
}
| nd=MINUS DIGIT+ DOT POSINTEGER {
ret = factory.realNumber(nd.getText());
}
| dbl=DOUBLE {
ret = factory.literal(Double.valueOf(dbl.getText()));
}
| ndbl='.' POSINTEGER Exponent | MINUS DIGIT+ DOT Exponent | MINUS DIGIT+
(DOT (POSINTEGER (Exponent)?)? | Exponent) {
ret = factory.literal(Double.valueOf(ndbl.getText()));
}
| NULL {ret = factory.literal(null);}
| TRUE {ret = factory.literal(Boolean.TRUE);}
| FALSE {ret = factory.literal(Boolean.FALSE);}
| dt=identifier t=STRING_LITERAL {
try {
ret = factory.literal(t.getText(), dt);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
;
{code}
was (Author: kliew):
Hi [~jamestaylor], I made the suggested changes to recreate the create-table
statement with the literal value string that we get after compiling the
statement.
Without a StatementContext, that takes us back to the issue with parsing
negative literals at `new SQLParser(column.getExpressionStr()).parseLiteral()`.
I tried defining a new parse function without adding new tokens so that other
functions won't be affected but the parser throws an exception.
{code}
// Get a string, integer, double, date, boolean, or NULL value.
// Able to parse negative numbers.
any_literal returns [LiteralParseNode ret]
: s=STRING_LITERAL {
ret = factory.literal(s.getText());
}
| n=NUMBER {
ret = factory.wholeNumber(n.getText());
}
| nn=MINUS DIGIT+ {
ret = factory.wholeNumber(nn.getText());
}
| d=DECIMAL {
ret = factory.realNumber(d.getText());
}
| nd=MINUS DIGIT+ DOT POSINTEGER {
ret = factory.realNumber(nd.getText());
}
| dbl=DOUBLE {
ret = factory.literal(Double.valueOf(dbl.getText()));
}
| ndbl='.' POSINTEGER Exponent | MINUS DIGIT+ DOT Exponent | MINUS DIGIT+
(DOT (POSINTEGER (Exponent)?)? | Exponent) {
ret = factory.literal(Double.valueOf(ndbl.getText()));
}
| NULL {ret = factory.literal(null);}
| TRUE {ret = factory.literal(Boolean.TRUE);}
| FALSE {ret = factory.literal(Boolean.FALSE);}
| dt=identifier t=STRING_LITERAL {
try {
ret = factory.literal(t.getText(), dt);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
;
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to
[jira] [Commented] (PHOENIX-476) Support declaration of DEFAULT in CREATE statement
[
https://issues.apache.org/jira/browse/PHOENIX-476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15597332#comment-15597332
]
Kevin Liew commented on PHOENIX-476:
Hi [~jamestaylor], I made the suggested changes to recreate the create-table
statement with the literal value string that we get after compiling the
statement.
Without a StatementContext, that takes us back to the issue with parsing
negative literals at `new SQLParser(column.getExpressionStr()).parseLiteral()`.
I tried defining a new parse function without adding new tokens so that other
functions won't be affected but the parser throws an exception.
{code}
// Get a string, integer, double, date, boolean, or NULL value.
// Able to parse negative numbers.
any_literal returns [LiteralParseNode ret]
: s=STRING_LITERAL {
ret = factory.literal(s.getText());
}
| n=NUMBER {
ret = factory.wholeNumber(n.getText());
}
| nn=MINUS DIGIT+ {
ret = factory.wholeNumber(nn.getText());
}
| d=DECIMAL {
ret = factory.realNumber(d.getText());
}
| nd=MINUS DIGIT+ DOT POSINTEGER {
ret = factory.realNumber(nd.getText());
}
| dbl=DOUBLE {
ret = factory.literal(Double.valueOf(dbl.getText()));
}
| ndbl='.' POSINTEGER Exponent | MINUS DIGIT+ DOT Exponent | MINUS DIGIT+
(DOT (POSINTEGER (Exponent)?)? | Exponent) {
ret = factory.literal(Double.valueOf(ndbl.getText()));
}
| NULL {ret = factory.literal(null);}
| TRUE {ret = factory.literal(Boolean.TRUE);}
| FALSE {ret = factory.literal(Boolean.FALSE);}
| dt=identifier t=STRING_LITERAL {
try {
ret = factory.literal(t.getText(), dt);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
;
{code}
> Support declaration of DEFAULT in CREATE statement
> --
>
> Key: PHOENIX-476
> URL: https://issues.apache.org/jira/browse/PHOENIX-476
> Project: Phoenix
> Issue Type: Task
>Affects Versions: 3.0-Release
>Reporter: James Taylor
>Assignee: Kevin Liew
> Labels: enhancement
> Fix For: 4.9.0
>
> Attachments: PHOENIX-476.2.patch, PHOENIX-476.3.patch,
> PHOENIX-476.4.patch, PHOENIX-476.5.patch, PHOENIX-476.6.patch,
> PHOENIX-476.7.patch, PHOENIX-476.patch
>
>
> Support the declaration of a default value in the CREATE TABLE/VIEW statement
> like this:
> CREATE TABLE Persons (
> Pid int NOT NULL PRIMARY KEY,
> LastName varchar(255) NOT NULL,
> FirstName varchar(255),
> Address varchar(255),
> City varchar(255) DEFAULT 'Sandnes'
> )
> To implement this, we'd need to:
> 1. add a new DEFAULT_VALUE key value column in SYSTEM.TABLE and pass through
> the value when the table is created (in MetaDataClient).
> 2. always set NULLABLE to ResultSetMetaData.columnNoNulls if a default value
> is present, since the column will never be null.
> 3. add a getDefaultValue() accessor in PColumn
> 4. for a row key column, during UPSERT use the default value if no value was
> specified for that column. This could be done in the PTableImpl.newKey method.
> 5. for a key value column with a default value, we can get away without
> incurring any storage cost. Although a little bit of extra effort than if we
> persisted the default value on an UPSERT for key value columns, this approach
> has the benefit of not incurring any storage cost for a default value.
> * serialize any default value into KeyValueColumnExpression
> * in the evaluate method of KeyValueColumnExpression, conditionally use
> the default value if the column value is not present. If doing partial
> evaluation, you should not yet return the default value, as we may not have
> encountered the the KeyValue for the column yet (since a filter evaluates
> each time it sees each KeyValue, and there may be more than one KeyValue
> referenced in the expression). Partial evaluation is determined by calling
> Tuple.isImmutable(), where false means it is NOT doing partial evaluation,
> while true means it is.
> * modify EvaluateOnCompletionVisitor by adding a visitor method for
> RowKeyColumnExpression and KeyValueColumnExpression to set
> evaluateOnCompletion to true if they have a default value specified. This
> will cause filter evaluation to execute one final time after all KeyValues
> for a row have been seen, since it's at this time we know we should use the
> default value.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
