[jira] [Updated] (PHOENIX-4971) Drop index will execute successfully using Incorrect name of parent tables
[ https://issues.apache.org/jira/browse/PHOENIX-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaanai updated PHOENIX-4971: Attachment: (was: PHOENIX-4971-master-v6.patch) > Drop index will execute successfully using Incorrect name of parent tables > -- > > Key: PHOENIX-4971 > URL: https://issues.apache.org/jira/browse/PHOENIX-4971 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.1 >Reporter: Jaanai >Assignee: Jaanai >Priority: Major > Fix For: 4.15.0 > > Attachments: PHOENIX-4971-master-v2.patch, > PHOENIX-4971-master-v3.patch, PHOENIX-4971-master-v4.patch, > PHOENIX-4971-master-v5-it-fixed.patch, PHOENIX-4971-master-v6.patch, > PHOENIX-4971-master.patch > > > The blew SQL will be executed successfully, the name of data tables is > incorrectly inputted that the parent table has the same name with index > tables. > > {code:java} > DROP INDEX INDEX_TABLE_X on INDEX_TABLE_X; > {code} > Some regions will not online and some queries will not normally execute after > executing above SQL. Everything will be ok unless manually delete dirty > metadata information in SYSTEM.CATALOG table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4971) Drop index will execute successfully using Incorrect name of parent tables
[ https://issues.apache.org/jira/browse/PHOENIX-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaanai updated PHOENIX-4971: Affects Version/s: 4.14.1 Fix Version/s: 4.15.0 > Drop index will execute successfully using Incorrect name of parent tables > -- > > Key: PHOENIX-4971 > URL: https://issues.apache.org/jira/browse/PHOENIX-4971 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.1 >Reporter: Jaanai >Assignee: Jaanai >Priority: Major > Fix For: 4.15.0 > > Attachments: PHOENIX-4971-master-v2.patch, > PHOENIX-4971-master-v3.patch, PHOENIX-4971-master-v4.patch, > PHOENIX-4971-master-v5-it-fixed.patch, PHOENIX-4971-master-v6.patch, > PHOENIX-4971-master.patch > > > The blew SQL will be executed successfully, the name of data tables is > incorrectly inputted that the parent table has the same name with index > tables. > > {code:java} > DROP INDEX INDEX_TABLE_X on INDEX_TABLE_X; > {code} > Some regions will not online and some queries will not normally execute after > executing above SQL. Everything will be ok unless manually delete dirty > metadata information in SYSTEM.CATALOG table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4971) Drop index will execute successfully using Incorrect name of parent tables
[ https://issues.apache.org/jira/browse/PHOENIX-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaanai updated PHOENIX-4971: Attachment: PHOENIX-4971-master-v6.patch > Drop index will execute successfully using Incorrect name of parent tables > -- > > Key: PHOENIX-4971 > URL: https://issues.apache.org/jira/browse/PHOENIX-4971 > Project: Phoenix > Issue Type: Bug >Reporter: Jaanai >Assignee: Jaanai >Priority: Major > Attachments: PHOENIX-4971-master-v2.patch, > PHOENIX-4971-master-v3.patch, PHOENIX-4971-master-v4.patch, > PHOENIX-4971-master-v5-it-fixed.patch, PHOENIX-4971-master-v6.patch, > PHOENIX-4971-master.patch > > > The blew SQL will be executed successfully, the name of data tables is > incorrectly inputted that the parent table has the same name with index > tables. > > {code:java} > DROP INDEX INDEX_TABLE_X on INDEX_TABLE_X; > {code} > Some regions will not online and some queries will not normally execute after > executing above SQL. Everything will be ok unless manually delete dirty > metadata information in SYSTEM.CATALOG table. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5042) Improve the exception message for local index to make it
[ https://issues.apache.org/jira/browse/PHOENIX-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Poonia updated PHOENIX-5042: - Attachment: PHOENIX-5042.patch > Improve the exception message for local index to make it > - > > Key: PHOENIX-5042 > URL: https://issues.apache.org/jira/browse/PHOENIX-5042 > Project: Phoenix > Issue Type: Improvement >Reporter: Aman Poonia >Priority: Minor > Attachments: PHOENIX-5042.patch > > > In one of the test we found that LOCAL INDEX Table doesn't support > HTableDescriptor as option in create statement. For instance if we do > {code:java} > CREATE local INDEX IDX_AAA_ORDERS ON AAA_ORDERS (o_custkey) INCLUDE > (o_orderkey, o_totalprice) DEFERRED_LOG_FLUSH=true;{code} > It throws an error message saying > {code:java} > Error: ERROR 1009 (42L02): Properties may not be defined for a view. > (state=42L02,code=1009){code} > This message doesn't tell how it is related to my local index!! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5042) Improve the exception message for local index to make it easy for user to understand
[ https://issues.apache.org/jira/browse/PHOENIX-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Poonia updated PHOENIX-5042: - Summary: Improve the exception message for local index to make it easy for user to understand (was: Improve the exception message for local index to make it ) > Improve the exception message for local index to make it easy for user to > understand > - > > Key: PHOENIX-5042 > URL: https://issues.apache.org/jira/browse/PHOENIX-5042 > Project: Phoenix > Issue Type: Improvement >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > Attachments: PHOENIX-5042.patch > > > In one of the test we found that LOCAL INDEX Table doesn't support > HTableDescriptor as option in create statement. For instance if we do > {code:java} > CREATE local INDEX IDX_AAA_ORDERS ON AAA_ORDERS (o_custkey) INCLUDE > (o_orderkey, o_totalprice) DEFERRED_LOG_FLUSH=true;{code} > It throws an error message saying > {code:java} > Error: ERROR 1009 (42L02): Properties may not be defined for a view. > (state=42L02,code=1009){code} > This message doesn't tell how it is related to my local index!! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PHOENIX-5042) Improve the exception message for local index to make it
[ https://issues.apache.org/jira/browse/PHOENIX-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Poonia reassigned PHOENIX-5042: Assignee: Aman Poonia > Improve the exception message for local index to make it > - > > Key: PHOENIX-5042 > URL: https://issues.apache.org/jira/browse/PHOENIX-5042 > Project: Phoenix > Issue Type: Improvement >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > Attachments: PHOENIX-5042.patch > > > In one of the test we found that LOCAL INDEX Table doesn't support > HTableDescriptor as option in create statement. For instance if we do > {code:java} > CREATE local INDEX IDX_AAA_ORDERS ON AAA_ORDERS (o_custkey) INCLUDE > (o_orderkey, o_totalprice) DEFERRED_LOG_FLUSH=true;{code} > It throws an error message saying > {code:java} > Error: ERROR 1009 (42L02): Properties may not be defined for a view. > (state=42L02,code=1009){code} > This message doesn't tell how it is related to my local index!! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-5042) Improve the exception message for local index to make it
Aman Poonia created PHOENIX-5042: Summary: Improve the exception message for local index to make it Key: PHOENIX-5042 URL: https://issues.apache.org/jira/browse/PHOENIX-5042 Project: Phoenix Issue Type: Improvement Reporter: Aman Poonia In one of the test we found that LOCAL INDEX Table doesn't support HTableDescriptor as option in create statement. For instance if we do {code:java} CREATE local INDEX IDX_AAA_ORDERS ON AAA_ORDERS (o_custkey) INCLUDE (o_orderkey, o_totalprice) DEFERRED_LOG_FLUSH=true;{code} It throws an error message saying {code:java} Error: ERROR 1009 (42L02): Properties may not be defined for a view. (state=42L02,code=1009){code} This message doesn't tell how it is related to my local index!! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5035) phoenix-spark dataframe filtes date or timestamp type with error
[ https://issues.apache.org/jira/browse/PHOENIX-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhongyuhai updated PHOENIX-5035: Attachment: (was: PHOENIX-5035.patch) > phoenix-spark dataframe filtes date or timestamp type with error > > > Key: PHOENIX-5035 > URL: https://issues.apache.org/jira/browse/PHOENIX-5035 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0, 4.14.0, 4.13.1, 5.0.0, 4.14.1 > Environment: HBase:apache 1.2 > Phoenix:4.13.1-HBase-1.2 > Hadoop:CDH 2.6 > Spark:2.3.1 >Reporter: zhongyuhai >Priority: Critical > Labels: patch, pull-request-available > Attachments: PHOENIX-5035.patch, table desc.png > > Original Estimate: 0h > Remaining Estimate: 0h > > *table desc as following:* > attach "table desc.png" > > *code as following:* > val df = SparkUtil.getActiveSession().read.format( > "org.apache.phoenix.spark").options(options).load() > df.filter("INCREATEDDATE = date'2018-07-14'") > > *exception as following:* > java.lang.RuntimeException: org.apache.phoenix.schema.TypeMismatchException: > ERROR 203 (22005): Type mismatch. DATE and BIGINT for "INCREATEDDATE" = 1997 > at > org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:201) > at > org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:87) > at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:127) > at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) > at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) > > *analyse as following:* > In the org.apache.phoenix.spark.PhoenixRelation.compileValue(value: Any): Any > , > > > {code:java} > private def compileValue(value: Any): Any = { > value match { > case stringValue: String => s"'${escapeStringConstant(stringValue)}'" > // Borrowed from 'elasticsearch-hadoop', support these internal UTF types > across Spark versions > // Spark 1.4 > case utf if (isClass(utf, "org.apache.spark.sql.types.UTF8String")) => > s"'${escapeStringConstant(utf.toString)}'" > // Spark 1.5 > case utf if (isClass(utf, "org.apache.spark.unsafe.types.UTF8String")) => > s"'${escapeStringConstant(utf.toString)}'" > > // Pass through anything else > case _ => value > } > {code} > > It only handles the String type , other type returns the toString。It makes > the Spark filte condition "INCREATEDDATE = date'2018-07-14'" translate to > Phoenix filte condition like "INCREATEDDATE = 2018-07-14" ,so Phoenix can not > run with this syntax and throw the exception ERROR 203 (22005): Type > mismatch. DATE and BIGINT for "INCREATEDDATE" = 1997 。 > *soluation as following:* > add handle to other type just like Date 、Timestamp > {code:java} > private def compileValue(value: Any): Any = { > value match { > case stringValue: String => s"'${escapeStringConstant(stringValue)}'" > // Borrowed from 'elasticsearch-hadoop', support these internal UTF types > across Spark versions > // Spark 1.4 > case utf if (isClass(utf, "org.apache.spark.sql.types.UTF8String")) => > s"'${escapeStringConstant(utf.toString)}'" > // Spark 1.5 > case utf if (isClass(utf, "org.apache.spark.unsafe.types.UTF8String")) => > s"'${escapeStringConstant(utf.toString)}'" > case d if(isClass(d , "java.lang.Date") || isClass(d , "java.sql.Date")) => { > val config: Configuration = > HBaseFactoryProvider.getConfigurationFactory.getConfiguration > val dateFormat = config.get(QueryServices.DATE_FORMAT_ATTRIB, > DateUtil.DEFAULT_DATE_FORMAT) > val df = new SimpleDateFormat(dateFormat) > s"date'${df.format(d)}'" > } > case dt if(isClass(dt , "java.sql.Timestamp")) => { > val config: Configuration = > HBaseFactoryProvider.getConfigurationFactory.getConfiguration > val dateTimeFormat = config.get(QueryServices.TIMESTAMP_FORMAT_ATTRIB, > DateUtil.DEFAULT_TIMESTAMP_FORMAT) > val df = new SimpleDateFormat(dateTimeFormat) > s"timestamp'${df.format(dt)}'" > } > // Pass through anything else > case _ => value > } > } > {code} > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)