[jira] [Commented] (PHOENIX-4609) Error Occurs while selecting a specific set of columns : ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had 2
[
https://issues.apache.org/jira/browse/PHOENIX-4609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378151#comment-16378151
]
Aman Jha commented on PHOENIX-4609:
---
[~ankit.singhal] can you comment on whether this issue is same as
-PHOENIX-4588- or any other fix is required?
> Error Occurs while selecting a specific set of columns : ERROR 201 (22000):
> Illegal data. Expected length of at least 8 bytes, but had 2
>
>
> Key: PHOENIX-4609
> URL: https://issues.apache.org/jira/browse/PHOENIX-4609
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0, 4.13.0
>Reporter: Aman Jha
>Priority: Critical
> Attachments: DML_DDL.sql, SelectStatement.sql, TestPhoenix.java
>
>
> While selecting columns from a table, an error occurs for Illegal Data.
> h3. _*ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes,
> but had 2*_
> The data is read/write only through the Phoenix Client.
> Moreover, this error only occurs while running queries via Java Program only
> and not through the Squirrel SQL client. Is there any other way to access
> results from the ResultSet that is returned from Phoenix Client.
>
> *Environment Details* :
> *HBase Version* : _1.2.6 on Hadoop 2.8.2_
> *Phoenix Version* : _4.11.0-HBase-1.2_
> *OS*: _LINUX(RHEL)_
>
> The following error is caused when selecting columns via a Java Program
> {code:java}
> ERROR 201 (22000): Illegal data. Expected length of at least 8 bytes, but had
> 2; nested exception is java.sql.SQLException: ERROR 201 (22000): Illegal
> data. Expected length of at least 8 bytes, but had 2
> at
> org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:102)
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
> at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
> at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419)
> at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474)
> at
> com.zycus.qe.service.impl.PhoenixHBaseDAOImpl.fetchAggregationResult(PhoenixHBaseDAOImpl.java:752)
> ... 14 common frames omitted
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected
> length of at least 8 bytes, but had 2
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:483)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
> at org.apache.phoenix.schema.KeyValueSchema.next(KeyValueSchema.java:213)
> at org.apache.phoenix.schema.KeyValueSchema.iterator(KeyValueSchema.java:165)
> at org.apache.phoenix.schema.KeyValueSchema.iterator(KeyValueSchema.java:171)
> at org.apache.phoenix.schema.KeyValueSchema.iterator(KeyValueSchema.java:175)
> at
> org.apache.phoenix.expression.ProjectedColumnExpression.evaluate(ProjectedColumnExpression.java:115)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:260)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:199)
> at
> org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.next(LookAheadResultIterator.java:65)
> at
> org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:255)
> at
> org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:199)
> at
> org.apache.phoenix.iterate.OrderedAggregatingResultIterator.next(OrderedAggregatingResultIterator.java:51)
> at
> org.apache.phoenix.iterate.DelegateResultIterator.next(DelegateResultIterator.java:44)
> at
> org.apache.phoenix.execute.TupleProjectionPlan$1.next(TupleProjectionPlan.java:62)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
> at
> org.apache.phoenix.iterate.LookAheadResultIterator.next(LookAheadResultIterator.java:65)
> at
>
[jira] [Commented] (PHOENIX-4622) Phoenix 4.13 order by issue
[
https://issues.apache.org/jira/browse/PHOENIX-4622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378149#comment-16378149
]
JeongMin Ju commented on PHOENIX-4622:
--
In version 4.13, the
[preStoreScannerOpen|https://github.com/apache/phoenix/blob/v4.13.2-cdh5.11.2/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java#L361-L375]
method has been added to the BaseScannerRegionObserver class.
This was the cause of the error.
Even if the plan is a reverse scan, a StoreScanner is created and returned. But
in case of reverse scan, ReverseStoreScanner should be returned.
This is fixed when the scan.isReversed () condition is added to the if
statement.
{code:java}
if (scan.isRaw() || scan.isReversed () ||
ScanInfoUtil.isKeepDeletedCells(store.getScanInfo()) ||
scan.getTimeRange().getMax() == HConstants.LATEST_TIMESTAMP ||
TransactionUtil.isTransactionalTimestamp(scan.getTimeRange().getMax())) {
return s;
}
{code}
I hope to be corrected soon.
I can not upgrade because of this problem.
> Phoenix 4.13 order by issue
> ---
>
> Key: PHOENIX-4622
> URL: https://issues.apache.org/jira/browse/PHOENIX-4622
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.13.1
> Environment: phoenix 4.13
> hbase 1.2.5
>Reporter: tom thmas
>Priority: Critical
>
> *1.create table and insert data.*
> create table test2
> (
> id varchar(200) primary key,
> cardid varchar(200),
> ctime date
> )
> upsert into test2 (id,cardid,ctime) values('a1','123',to_date('2017-12-01
> 17:42:45'))
> *2.query sql like this:*
> select id,ctime from test2 where cardid='123' order by ctime
> error log:
> {color:#FF}org.apache.phoenix.exception.PhoenixIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> TEST2,,1519221167250.813e4ce0510965a7a7898413da2a17ad.: null{color}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4505) IndexHalfStoreFileReaderGenerator.preStoreScannerOpen does not close the passed in KeyValueScanner while creating and returning a new scanner
[ https://issues.apache.org/jira/browse/PHOENIX-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378119#comment-16378119 ] James Taylor commented on PHOENIX-4505: --- [~sanjeevln] - this should be fixed too. > IndexHalfStoreFileReaderGenerator.preStoreScannerOpen does not close the > passed in KeyValueScanner while creating and returning a new scanner > - > > Key: PHOENIX-4505 > URL: https://issues.apache.org/jira/browse/PHOENIX-4505 > Project: Phoenix > Issue Type: Bug >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva >Priority: Major > > [~jamestaylor] > preCompactScannerOpen also does not close the InternalScanner that is passed > in before creating and returning a new scanner, does that scanner need to be > closed as well? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377979#comment-16377979
]
Jepson commented on PHOENIX-4629:
-
Add the parameter to hbase-site.xml, is not work.
{code:java}
phoenix.query.dateFormatTimeZone
Asia/Shanghai
{code}
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Attachments: Phoenix-4629.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377969#comment-16377969
]
Jepson commented on PHOENIX-4629:
-
Compile the jar, test is also ok.
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Attachments: Phoenix-4629.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated PHOENIX-4629:
Attachment: Phoenix-4629.patch
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Attachments: Phoenix-4629.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated PHOENIX-4629:
External issue ID: PHOENIX-3221 PHOENIX-997 /PHOENIX-1485 (was:
PHOENIX-3221 )
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated PHOENIX-4629:
External issue ID: PHOENIX-3221 PHOENIX-997 PHOENIX-1485 (was:
PHOENIX-3221 PHOENIX-997 /PHOENIX-1485)
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-4629) timestamp with timezone issue
[
https://issues.apache.org/jira/browse/PHOENIX-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jepson updated PHOENIX-4629:
External issue ID: PHOENIX-3221 (was: 3221)
> timestamp with timezone issue
> -
>
> Key: PHOENIX-4629
> URL: https://issues.apache.org/jira/browse/PHOENIX-4629
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.10.0
> Environment: phoenix4.10-hbase1.2
>Reporter: Jepson
>Priority: Major
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> *1.Create timezonetest table:*
> {code:java}
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
> *2.Create TimestampTest.java*
> {code:java}
> package org.apache.phoenix.jdbc;
> import org.apache.phoenix.query.BaseConnectionlessQueryTest;
> import org.apache.phoenix.query.QueryServices;
> import java.sql.Connection;
> import java.sql.DriverManager;
> import java.sql.ResultSet;
> import java.util.Properties;
> /**
> * Created by Jepson on 2017/11/2.
> *
> CREATE TABLE JYDW.timezonetest (
> id bigint(11) not null primary key,
> date_c date ,
> datetime_c timestamp ,
> timestamp_c timestamp
> )SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
> */
> public class TimestampTest extends BaseConnectionlessQueryTest {
> public static void main(String[] args) throws Exception {
> Properties props = new Properties();
>// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
> "Asia/Shanghai");
> String url =
> "jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
> //Connection conn = DriverManager.getConnection(url,props);
> Connection conn = DriverManager.getConnection(url);
> conn.createStatement().execute("UPSERT INTO
> jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
> "values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
> 10:00:00')");
> conn.commit();
> ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
> TIMEZONETEST");
> while(rs.next()) {
> System.out.println(rs.getString("id")+" : " +
> rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
> rs.getString("timestamp_c"));
> }
> rs.close();
> conn.close();
> }
> }
> {code}
> *3.Run the TimestampTest.java,the console print message:*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> 100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
> 02:00:00.000
> *{color:#ff}minus 8 hours, is also wrong.{color}*
> *4.Reference these, not work*
> https://issues.apache.org/jira/browse/PHOENIX-997
> https://issues.apache.org/jira/browse/PHOENIX-1485
> 5.Modify DateUtil.java
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "GMT";
> public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
> *Changed:*
> {code:java}
> public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
> public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
> {code}
> -
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
> *Changed:*
> {code:java}
> private final DateTimeFormatter formatter =
> ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
> {code}
>
> 6.Again run *TimestampTest.java, the result is ok.*
> *id : date_c : datetime_c : timestamp_c*
> 101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
> 100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
> 10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-4629) timestamp with timezone issue
Jepson created PHOENIX-4629:
---
Summary: timestamp with timezone issue
Key: PHOENIX-4629
URL: https://issues.apache.org/jira/browse/PHOENIX-4629
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.10.0
Environment: phoenix4.10-hbase1.2
Reporter: Jepson
*1.Create timezonetest table:*
{code:java}
CREATE TABLE JYDW.timezonetest (
id bigint(11) not null primary key,
date_c date ,
datetime_c timestamp ,
timestamp_c timestamp
)SALT_BUCKETS = 12, COMPRESSION='SNAPPY';{code}
*2.Create TimestampTest.java*
{code:java}
package org.apache.phoenix.jdbc;
import org.apache.phoenix.query.BaseConnectionlessQueryTest;
import org.apache.phoenix.query.QueryServices;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.util.Properties;
/**
* Created by Jepson on 2017/11/2.
*
CREATE TABLE JYDW.timezonetest (
id bigint(11) not null primary key,
date_c date ,
datetime_c timestamp ,
timestamp_c timestamp
)SALT_BUCKETS = 12, COMPRESSION='SNAPPY';
*/
public class TimestampTest extends BaseConnectionlessQueryTest {
public static void main(String[] args) throws Exception {
Properties props = new Properties();
// props.setProperty(QueryServices.DATE_FORMAT_TIMEZONE_ATTRIB,
"Asia/Shanghai");
String url =
"jdbc:phoenix:192.168.117.137,192.168.117.138,192.168.117.140,192.168.117.141,192.168.117.142:2181:/hbase";
//Connection conn = DriverManager.getConnection(url,props);
Connection conn = DriverManager.getConnection(url);
conn.createStatement().execute("UPSERT INTO
jydw.TIMEZONETEST(id,date_c,datetime_c,timestamp_c) \n" +
"values(101,'2018-02-25','2018-02-25 00:00:00','2018-02-25
10:00:00')");
conn.commit();
ResultSet rs = conn.createStatement().executeQuery("SELECT * FROM
TIMEZONETEST");
while(rs.next()) {
System.out.println(rs.getString("id")+" : " +
rs.getString("date_c")+" : " + rs.getString("datetime_c")+" : " +
rs.getString("timestamp_c"));
}
rs.close();
conn.close();
}
}
{code}
*3.Run the TimestampTest.java,the console print message:*
*id : date_c : datetime_c : timestamp_c*
101 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
02:00:00.000
100 : 2018-02-24 16:00:00.000 : 2018-02-24 16:00:00.000 : 2018-02-25
02:00:00.000
*{color:#ff}minus 8 hours, is also wrong.{color}*
*4.Reference these, not work*
https://issues.apache.org/jira/browse/PHOENIX-997
https://issues.apache.org/jira/browse/PHOENIX-1485
5.Modify DateUtil.java
{code:java}
public static final String DEFAULT_TIME_ZONE_ID = "GMT";
public static final String LOCAL_TIME_ZONE_ID = "LOCAL";{code}
*Changed:*
{code:java}
public static final String DEFAULT_TIME_ZONE_ID = "Asia/Shanghai";
public static final String LOCAL_TIME_ZONE_ID = "Asia/Shanghai";
{code}
-
{code:java}
private final DateTimeFormatter formatter =
ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("UTC"));{code}
*Changed:*
{code:java}
private final DateTimeFormatter formatter =
ISO_DATE_TIME_FORMATTER.withZone(DateTimeZone.forID("Asia/Shanghai"));
{code}
6.Again run *TimestampTest.java, the result is ok.*
*id : date_c : datetime_c : timestamp_c*
101 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
10:00:00.000
100 : 2018-02-25 00:00:00.000 : 2018-02-25 00:00:00.000 : 2018-02-25
10:00:00.000
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-1890) Provide queries for adding/deleting jars to/from common place in hdfs which is used by dynamic class loader
[ https://issues.apache.org/jira/browse/PHOENIX-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377823#comment-16377823 ] Ethan Wang commented on PHOENIX-1890: - [~rajeshbabu] I see. and it is not supporting something like: add jars hdfs://localhost:9000/dir/a.jar correct? > Provide queries for adding/deleting jars to/from common place in hdfs which > is used by dynamic class loader > --- > > Key: PHOENIX-1890 > URL: https://issues.apache.org/jira/browse/PHOENIX-1890 > Project: Phoenix > Issue Type: Sub-task >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Fix For: 4.5.0 > > Attachments: PHOENIX-1890.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-4628) Allow min time between update stats to be configurable separately from stats cache TTL
James Taylor created PHOENIX-4628: - Summary: Allow min time between update stats to be configurable separately from stats cache TTL Key: PHOENIX-4628 URL: https://issues.apache.org/jira/browse/PHOENIX-4628 Project: Phoenix Issue Type: Bug Reporter: James Taylor Assignee: James Taylor Fix For: 4.14.0 We only have a single default config that controls both how long stats are cached and how often we allow UPDATE STATISTICS to be called. We should have separate property values for those two distinct things. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4626) Increase time allowed for partial index rebuild to complete
[ https://issues.apache.org/jira/browse/PHOENIX-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377796#comment-16377796 ] Thomas D'Silva commented on PHOENIX-4626: - +1 > Increase time allowed for partial index rebuild to complete > --- > > Key: PHOENIX-4626 > URL: https://issues.apache.org/jira/browse/PHOENIX-4626 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4626_v1.patch > > > Currently a mutable index is marked as disabled if it cannot be caught up by > the partial index rebuilder after 30 minutes. This is too short a time. > Instead, we should allow 24 hours. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4626) Increase time allowed for partial index rebuild to complete
[ https://issues.apache.org/jira/browse/PHOENIX-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377791#comment-16377791 ] James Taylor commented on PHOENIX-4626: --- Simple patch to change default config. Please review, [~tdsilva]. > Increase time allowed for partial index rebuild to complete > --- > > Key: PHOENIX-4626 > URL: https://issues.apache.org/jira/browse/PHOENIX-4626 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4626_v1.patch > > > Currently a mutable index is marked as disabled if it cannot be caught up by > the partial index rebuilder after 30 minutes. This is too short a time. > Instead, we should allow 24 hours. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4626) Increase time allowed for partial index rebuild to complete
[ https://issues.apache.org/jira/browse/PHOENIX-4626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4626: -- Attachment: PHOENIX-4626_v1.patch > Increase time allowed for partial index rebuild to complete > --- > > Key: PHOENIX-4626 > URL: https://issues.apache.org/jira/browse/PHOENIX-4626 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor >Priority: Major > Fix For: 4.14.0 > > Attachments: PHOENIX-4626_v1.patch > > > Currently a mutable index is marked as disabled if it cannot be caught up by > the partial index rebuilder after 30 minutes. This is too short a time. > Instead, we should allow 24 hours. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4333) Incorrect estimate when stats are updated on a tenant specific view
[
https://issues.apache.org/jira/browse/PHOENIX-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377752#comment-16377752
]
Thomas D'Silva commented on PHOENIX-4333:
-
+1 LGTM
> Incorrect estimate when stats are updated on a tenant specific view
> ---
>
> Key: PHOENIX-4333
> URL: https://issues.apache.org/jira/browse/PHOENIX-4333
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Major
> Labels: SFDC, stats
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4333_test.patch, PHOENIX-4333_v1.patch,
> PHOENIX-4333_v2.patch, PHOENIX-4333_v3.patch, PHOENIX-4333_wip1.patch,
> PHOENIX-4333_wip2.patch, PHOENIX-4333_wip3.patch, PHOENIX-4333_wip4.patch
>
>
> Consider two tenants A, B with tenant specific view on 2 separate
> regions/region servers.
> {noformat}
> Region 1 keys:
> A,1
> A,2
> B,1
> Region 2 keys:
> B,2
> B,3
> {noformat}
> When stats are updated on tenant A view. Querying stats on tenant B view
> yield partial results (only contains stats for B,1) which are incorrect even
> though it shows updated timestamp as current.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PHOENIX-4231) Support restriction of remote UDF load sources
[
https://issues.apache.org/jira/browse/PHOENIX-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377696#comment-16377696
]
ASF GitHub Bot commented on PHOENIX-4231:
-
Github user aertoria commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/292#discussion_r170755367
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java ---
@@ -907,10 +909,15 @@ public MutationState execute() throws SQLException {
try {
FileSystem fs =
dynamicJarsDirPath.getFileSystem(conf);
List jarPaths = getJarPaths();
-for (LiteralParseNode jarPath : jarPaths) {
-File f = new File((String) jarPath.getValue());
-fs.copyFromLocalFile(new
Path(f.getAbsolutePath()), new Path(
-dynamicJarsDir + f.getName()));
+for (LiteralParseNode jarPathNode : jarPaths) {
+ String jarPathName = (String)
jarPathNode.getValue();
+ File f = new File(jarPathName);
+ Path dynamicJarsDirPathWithJar = new
Path(dynamicJarsDir + f.getName());
+ // Copy the jar (can be local or on HDFS) to the
hbase.dynamic.jars.dir directory.
+ // Note that this does not support HDFS URIs
without scheme and authority.
+ Path jarPath = new Path(jarPathName);
+ FileUtil.copy(jarPath.getFileSystem(conf),
jarPath, fs, dynamicJarsDirPathWithJar,
--- End diff --
The comment "Note that this does not support HDFS URIs without scheme and
authority." I wonder where did you discover that.
If I'm not mistaken FileUtil.copy internally calls fs.copyFromLocalFile
without doing file.islocal check.
> Support restriction of remote UDF load sources
> ---
>
> Key: PHOENIX-4231
> URL: https://issues.apache.org/jira/browse/PHOENIX-4231
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Chinmay Kulkarni
>Priority: Major
>
> When allowUserDefinedFunctions is true, users can load UDFs remotely via a
> jar file from any HDFS filesystem reachable on the network. The setting
> hbase.dynamic.jars.dir can be used to restrict locations for jar loading but
> is only applied to jars loaded from the local filesystem. We should
> implement support for similar restriction via configuration for jars loaded
> via hdfs:// URIs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] phoenix pull request #292: PHOENIX-4231: Support restriction of remote UDF l...
Github user aertoria commented on a diff in the pull request:
https://github.com/apache/phoenix/pull/292#discussion_r170755367
--- Diff:
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java ---
@@ -907,10 +909,15 @@ public MutationState execute() throws SQLException {
try {
FileSystem fs =
dynamicJarsDirPath.getFileSystem(conf);
List jarPaths = getJarPaths();
-for (LiteralParseNode jarPath : jarPaths) {
-File f = new File((String) jarPath.getValue());
-fs.copyFromLocalFile(new
Path(f.getAbsolutePath()), new Path(
-dynamicJarsDir + f.getName()));
+for (LiteralParseNode jarPathNode : jarPaths) {
+ String jarPathName = (String)
jarPathNode.getValue();
+ File f = new File(jarPathName);
+ Path dynamicJarsDirPathWithJar = new
Path(dynamicJarsDir + f.getName());
+ // Copy the jar (can be local or on HDFS) to the
hbase.dynamic.jars.dir directory.
+ // Note that this does not support HDFS URIs
without scheme and authority.
+ Path jarPath = new Path(jarPathName);
+ FileUtil.copy(jarPath.getFileSystem(conf),
jarPath, fs, dynamicJarsDirPathWithJar,
--- End diff --
The comment "Note that this does not support HDFS URIs without scheme and
authority." I wonder where did you discover that.
If I'm not mistaken FileUtil.copy internally calls fs.copyFromLocalFile
without doing file.islocal check.
---
[DISCUSS] Design for a "query log"
Hiya, I wanted to share this little design doc with you about some feature work we've been thinking about. The following is a Google doc in which anyone should be allowed to comment. Feel free to comment there, or here on the thread. https://s.apache.org/phoenix-query-log The high-level goal is to create a construct in which Phoenix clients will automatically serialize information about the queries they run to a table for retrospective analysis. Ideally, this information would be stored in a Phoenix table. We want this data to help answer questions like: * What queries are running against my system * What specific queries started between 535AM and 620AM two days ago * What queries are user "bob" running * Are my user's queries effectively using the indexes in the system Anti-goals for include: * Cluster impact (computation/memory) usage of a query * Query performance may be slowed to ensure all data is serialized * A third-party service dedicated to ensuring query info is serialized (in the event of client failure) Take a look at the document and let us know what you think please. I'm happy to try to explain this in greater detail. - Josh (on behalf of myself, Ankit, Rajeshbabu, and Sergey)
[jira] [Commented] (PHOENIX-4231) Support restriction of remote UDF load sources
[ https://issues.apache.org/jira/browse/PHOENIX-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377520#comment-16377520 ] Ethan Wang commented on PHOENIX-4231: - To summarize what we are going to be working on this jira: As of today, loading from add jars 'XX' allows loading the jars from any hdfs network location to local's hbase.dynamic.jars.dir. So we are now going to make addition features to PHOENIX-1890 so that: when a property, say "loading.from.network.allowed" property is configured as false, "add jars" command only restricted to load from local file system or local hdfs. Is the plan above accurate? [~rajeshbabu] [~apurtell] Once this feature is done. user will be doing the following for creating a Udf: Add jars hdfs://localhost/udf.jar CREATE FUNCTION func(VARCHAR) returns VARCHAR as 'com.example.test.udfFunc' using jar 'configured/hbase/dynamic/jars/dir/udf2.jar' > Support restriction of remote UDF load sources > --- > > Key: PHOENIX-4231 > URL: https://issues.apache.org/jira/browse/PHOENIX-4231 > Project: Phoenix > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Chinmay Kulkarni >Priority: Major > > When allowUserDefinedFunctions is true, users can load UDFs remotely via a > jar file from any HDFS filesystem reachable on the network. The setting > hbase.dynamic.jars.dir can be used to restrict locations for jar loading but > is only applied to jars loaded from the local filesystem. We should > implement support for similar restriction via configuration for jars loaded > via hdfs:// URIs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-4607) Allow PhoenixInputFormat to use tenant-specific connections
[ https://issues.apache.org/jira/browse/PHOENIX-4607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377484#comment-16377484 ] Geoffrey Jacoby commented on PHOENIX-4607: -- [~jamestaylor] [~mujtabachohan] [~tdsilva], if one of you has bandwidth could I please get a review? Thanks! > Allow PhoenixInputFormat to use tenant-specific connections > --- > > Key: PHOENIX-4607 > URL: https://issues.apache.org/jira/browse/PHOENIX-4607 > Project: Phoenix > Issue Type: New Feature >Affects Versions: 4.13.0 >Reporter: Geoffrey Jacoby >Assignee: Geoffrey Jacoby >Priority: Major > Labels: mapreduce > Attachments: PHOENIX-4607.patch > > > When using Phoenix's MapReduce integration, the actual connections for the > SELECT query are created by PhoenixInputFormat. While PhoenixInputFormat has > support for a few connection properties such as SCN, a TenantId is not one of > them. > Add the ability to specify a TenantId for the PhoenixInputFormat's > connections to use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (PHOENIX-1890) Provide queries for adding/deleting jars to/from common place in hdfs which is used by dynamic class loader
[
https://issues.apache.org/jira/browse/PHOENIX-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376964#comment-16376964
]
Rajeshbabu Chintaguntla commented on PHOENIX-1890:
--
[~aertoria] correct it's not documented in grammar. Will add them to
documentation.
{noformat}
add jars (jarpath[,jarpath])
{noformat}
{noformat}
delete jar
{noformat}
{noformat}
list jars
{noformat}
> Provide queries for adding/deleting jars to/from common place in hdfs which
> is used by dynamic class loader
> ---
>
> Key: PHOENIX-1890
> URL: https://issues.apache.org/jira/browse/PHOENIX-1890
> Project: Phoenix
> Issue Type: Sub-task
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 4.5.0
>
> Attachments: PHOENIX-1890.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
