Re: Indexes not used when ordering by primary key.

2017-10-14 Thread James Taylor
Couple of follow up comments: - if you use c1=‘X0’ the index should be used without a hint, because it’s still ordered by the PK when using index. - this wouldn’t necessarily be the case for c1 LIKE 'X0%'. On Fri, Oct 13, 2017 at 8:33 PM James Taylor <jamestay...@apache.org> wrote:

Re: Indexes not used when ordering by primary key.

2017-10-13 Thread James Taylor
Yes, this is expected behavior. Phoenix can either optimize based on the filter in the WHERE clause or the ORDER BY. Since it's not cost based, Phoenix always chooses to optimize out the ORDER BY (because in general it's more expensive and it doesn't know how much data will be filtered out by the

[ANNOUNCE] Apache Phoenix 4.12 released

2017-10-11 Thread James Taylor
The Apache Phoenix team is pleased to announce the immediate availability of the 4.12.0 release [1]. Apache Phoenix enables SQL-based OLTP and operational analytics for Apache Hadoop using Apache HBase as its backing store and providing integration with other projects in the Apache ecosystem such

Re: Async get

2017-10-05 Thread James Taylor
gle.ca/amp/blog.colinbreck.com/calling-blocki >> ng-code-there-is-no-free-lunch/amp/ >> >> On Wed, Oct 4, 2017 at 11:36 AM Stan Campbell <stan.campbe...@gmail.com> >> wrote: >> >>> Wrap the call in a Future. You're home. >>> >>> On Wed, Oc

Re: Async get

2017-10-04 Thread James Taylor
Hi Flavio, Phoenix supports JDBC. The implementation may do gets, scans, etc., but it's completely transparent to the user. Thanks, James On Wed, Oct 4, 2017 at 6:36 AM, Flavio Pompermaier wrote: > Hi to all, > does Phoenix support async get? I can't find anything about

Re: using index with "or" query

2017-10-03 Thread James Taylor
rget-temp table (after firing all queries) , and >drop temp-table > > though this approach results in firing more queries, it can complete in > less time than firing queries without using index > > On Wed, Sep 20, 2017 at 9:31 PM, James Taylor <jamestay...@apache.org> >

Re: Phoenix join on derived table and documentation

2017-09-25 Thread James Taylor
Ryan - did the work around solve the issue? Thanks, James On Fri, Sep 22, 2017 at 11:22 AM, Maryann Xue wrote: > Hi Ryan, > > Could you please try: > > select > > MAIN.EBELN as PO_DOC, > > MAIN.EBELP as PO_DOC_ITEM_NUM, > > XREF.DELV_DOC > > from EKPO as MAIN > > left

Re: [ANNOUNCE] New PMC Member: Sergey Soldatov

2017-09-25 Thread James Taylor
Great to have you on the PMC, Sergey. Congrats! James On Mon, Sep 25, 2017 at 10:28 AM, Josh Mahonin wrote: > Congratulations Sergey! > > On Sun, Sep 24, 2017 at 4:05 PM, Ted Yu wrote: > > > Congratulations, Sergey ! > > > > On Sun, Sep 24, 2017 at

Re: using index with "or" query

2017-09-20 Thread James Taylor
tiple indexes? > Hint on local index cause an error of unknown field if the where clause > refer to a field in the index and to another field that is not part of the > specific local index. There is an open bug on it. > > BR, > > Yoav Sapir > > > On 19 Sep 2017, at 18:21,

Re: using index with "or" query

2017-09-19 Thread James Taylor
Hi Noam, A few ideas: 1) Use a UNION instead of an OR and you may be able to use more than one index for one query. 2) Include the columns you're referencing in the index to make it a covered index [1]. 3) Hint [2] the query to force the index to be used. Thanks, James [1]

Re: Salt Number

2017-09-08 Thread James Taylor
Hi Pradheep, Would you be able to describe your use case and why you're salting? We really only recommend salting if you have write hotspotting. Otherwise, it increases the overall load on your cluster. Thanks, James On Fri, Sep 8, 2017 at 9:13 AM, Pradheep Shanmugam <

Re: How to speed up write performance

2017-09-08 Thread James Taylor
write performance raised to 1k/s with the >> tuning above. >> I also tried using multi-thread for write, with 20 threads write >> concurrently, the performance can raise to 5k/s, but won't raise any more >> how ever threads increased. >> >> I didn't try wr

Re: How to speed up write performance

2017-09-06 Thread James Taylor
Hi Hef, Have you had a chance to read our Tuning Guide [1] yet? There's a lot of good, general guidance there. There are some optimizations for write performance that depend on how you expect/allow your data and schema to change: 1) Is your data write-once? Make sure to declare your table with the

Re: optimistic lock in phenix

2017-08-30 Thread James Taylor
Another option is to upgrade to 4.8 or later and use transactions (which uses optimistic concurrency under the covers): https://phoenix.apache.org/transactions.html Thanks, James On Wed, Aug 30, 2017 at 9:53 AM, James Taylor <jamestay...@apache.org> wrote: > Upgrade to 4.9 or later an

Re: optimistic lock in phenix

2017-08-30 Thread James Taylor
Upgrade to 4.9 or later and use our atomic upsert command: http://phoenix.apache.org/atomic_upsert.html On Wed, Aug 30, 2017 at 9:38 AM Pradheep Shanmugam < pradheep.shanmu...@infor.com> wrote: > hi, > > > we have a table in phoenix 4.4 which has a modifyrevision as a column.. > > when a thread

Re: How to specify the point lookup chunks number(parallelism) at query

2017-08-29 Thread James Taylor
The finest granularity we have available is to specify a guidepost width at the table level with the GUIDEPOSTS_WIDTH=## property. This will take effect for that table next time stats are collected (on a major compaction or when collected manually through the UPDATE STATISTICS command). On Tue,

Re: Duplicate rows with the PK

2017-08-28 Thread James Taylor
Is the table salted? If the SALT_BUCKETS property was ever changed during the life of the table, this could lead to duplicates. You could get the same row key with a different salt byte. If not salted, are there any global, mutable secondary indexes. A number of issues have been fixed which will

Re: Delay between put from HBase shell and result in SELECT from Phoenix

2017-08-25 Thread James Taylor
Phoenix retrieves the server timestamp from the region server that hosts the system catalog table and uses that as the timestamp of the puts when you do an UPSERT VALUE (FYI, this behavior will change in 4.12 and we'll use latest timestamp everywhere). I suspect the puts you're doing are going to

Re: quick question about apache phoenix.

2017-08-21 Thread James Taylor
Hi Pawan, Why would you listen to someone about the future of Apache Phoenix who has no involvement in or knowledge about the project? Michael Segel was banned on the HBase lists for a while due to being too belligerent. He's not a committer on any Apache project. Has he even contributed a patch?

Re: Full Scan - is it really?

2017-07-27 Thread James Taylor
The query for products is doing a point lookup based on the combinations of product_id, sha_key, zip_code and then (I think) a skip scan is being done for clicks based on the sha_key (based on the DYNAMIC SERVER FILTER). You should be able to confirm this in a debugger through the scans that take

Re: Full Scan - is it really?

2017-07-27 Thread James Taylor
I agree - the explain plan is confusing. Would be good to capture fixing that in a JIRA. Would you mind sharing your DDL and queries too? On Thu, Jul 27, 2017 at 3:55 PM, Josh Elser wrote: > > > On 7/27/17 4:36 PM, Lew Jackman wrote: > >> I am joining two tables by using only

Re: Can set default value for column in phoenix ?

2017-07-14 Thread James Taylor
Only constant expressions are allowed for default value currently (as mentioned by Juvenn). PHOENIX-3425 is for supporting expressions such as CURRENT_DATE and NEXT VALUE FOR. On Fri, Jul 14, 2017 at 7:22 AM Ankit Singhal wrote: > Phoenix 4.9 onwards you can specify

[ANNOUNCE] Apache Phoenix 4.11 released

2017-07-07 Thread James Taylor
The Apache Phoenix team is pleased to announce the immediate availability of the 4.11.0 release. Apache Phoenix enables SQL-based OLTP and operational analytics for Apache Hadoop using Apache HBase as its backing store and providing integration with other projects in the Apache ecosystem such as

Re: how to query data in index table?

2017-07-05 Thread James Taylor
You shouldn't need to query an index table directly as Phoenix will use an index automatically based on a determination by the optimizer that it's more efficient to do so. If for diagnostic purposes you need to query the index table directly (or any table that uses special characters in the column

Re: Getting too many open files during table scan

2017-06-22 Thread James Taylor
My recommendation: don't use salt buckets unless you have a monatomically increasing row key, for example one that leads with the current date/time. Otherwise you'll be putting more load (# of salt buckets more load worst case) for bread-and-butter small-range-scan Phoenix queries. Thanks, James

Re: Explain Plan - Full scan vs Skip scan

2017-06-22 Thread James Taylor
Hi Michael, Would it be possible for you to provide an example with DDL, query, and explain plan? Thanks, James On Fri, Jun 23, 2017 at 2:36 AM Mike Prendergast wrote: > We are currently using Phoenix 4.9 in EMR 5.6. We have been constructing > queries that we would

Re: Best strategy for UPSERT SELECT in large table

2017-06-16 Thread James Taylor
Hi Pedro, Before 4.10, it will be single client (though multi threaded). With 4.10 and above, the statement would run distributed across your cluster so performance should improve. Note that if the source table is taking writes while the UPSERT SELECT is running, the statement would miss those

Re: Phoenix index update on direct HBase row update

2017-06-15 Thread James Taylor
point me to example or to related > source code of Phoenix API? > > On 16 Jun 2017, at 03:41, James Taylor <jamestay...@apache.org> wrote: > > No, not unless you write the code that keeps the index in sync yourself, > attach the metadata needed by index maintenance coproces

Re: Phoenix index update on direct HBase row update

2017-06-15 Thread James Taylor
No, not unless you write the code that keeps the index in sync yourself, attach the metadata needed by index maintenance coprocessor in your update code, or use the Phoenix update APIs which do this for you. On Thu, Jun 15, 2017 at 5:29 PM, Batyrshin Alexander <0x62...@gmail.com> wrote: > We

Re: Partial indexes

2017-06-08 Thread James Taylor
Hi Alexander, We support indexes on views which is essentially what you're asking for. Thanks, James On Thu, Jun 8, 2017 at 1:28 AM Batyrshin Alexander <0x62...@gmail.com> wrote: > Hello, > Is there any plans to implement partial indexes (CREATE INDEX WHERE > predicate)?

Re: Delete from Array

2017-06-06 Thread James Taylor
be either "index" or "value" (what to delete by) > and the last would be the index or value to delete > > Regards, > Cheyenne > > On Tue, Jun 6, 2017 at 6:17 PM, James Taylor <jamestay...@apache.org> > wrote: > >> Please feel free to file

Re: Scan phoenix created columns, hbase

2017-06-05 Thread James Taylor
We don't have APIs for what you're trying to do. Our APIs are JDBC. For these low level kinds of things, I'd recommend reading and understanding the source code. Thanks, James On Mon, Jun 5, 2017 at 1:09 PM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > Can anyone please help? > >

Re: Short Tables names and column names

2017-05-31 Thread James Taylor
Thanks, Ash. We welcome any doc enhancements. See http://phoenix.apache.org/building_website.html on how to do this. James On Wed, May 31, 2017 at 7:54 AM, Ash N <742...@gmail.com> wrote: > Hi Samarth, > > Your comment is quite decisive and actionable. > > I'd like to edit the documentation

Re: secondary index consistency guarantee

2017-05-31 Thread James Taylor
Hello, The index rebuild process is incremental, based on the time stamp at which the index write failure originally occurred. Thanks, James On Wed, May 31, 2017 at 2:22 AM 金砖 wrote: > hi, everyone: > > I read from document there are 3 levels of consistency. > >

Re: NPE when filtering on TIMESTAMP

2017-05-23 Thread James Taylor
Answered over on SO On Tue, May 23, 2017 at 3:34 PM, Barrett Strausser wrote: > Crossposted to SO > > https://stackoverflow.com/questions/44144925/apache- > phoenix-current-time-gives-npe >

Re: Phoenix hbase question

2017-05-23 Thread James Taylor
FWIW, we're exposing a way to do snapshot reads (PHOENIX-3744), starting with our MR integration (on top of which the Spark integration is built) for our 4.11 release. This is about as close as you can get to reading HDFS directly while still taking into account non flushed HBase data. Thanks,

Re: Upgrade Phoenix version on HDP 2.4

2017-05-19 Thread James Taylor
But hopefully Hortonworks hears (from you and many others) that you'd like to see HDP move to a newer version of Phoenix. The 4.7 release is four releases back from the current 4.10 release. Any ballpark timeframe for an HDP release with the latest Phoenix release? On Fri, May 19, 2017 at 7:56

Re: pagination

2017-05-18 Thread James Taylor
t; An d keep the offsets for each page in the server side, for a while... > (the client might not need all of them, might also never ask for next page) > > http://www.oracle.com/technetwork/java/valuelisthandler-142464.html > > On May 18, 2017 20:02, "James Taylor" <james

Re: pagination

2017-05-18 Thread James Taylor
Yes, it's expected that query performance would degrade as the offset increases. The best Phoenix can do for OFFSET is to scan the rows and count them until the offset is reached. Use the row value constructor technique instead to prevent this: https://phoenix.apache.org/paged.html On Thu, May

Re: Sequences vs UUID's

2017-05-05 Thread James Taylor
Hi Ash, Typically users come up with a composite primary key based on event data that is unique. For example: . If you don't have an event ID from an external source, perhaps you can either use a cyclical sequence or a random number (using our RAND built-in function). Would that give you the

Re: Unable to delete from table

2017-04-28 Thread James Taylor
Did you commit the data? I'd recommend turning auto commit on for the connection before running the delete statement. Thanks, James On Fri, Apr 28, 2017 at 12:36 PM, Michael Young wrote: > This is probably a newbie question, but has anyone run across a case where > a

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread James Taylor
; >>>>> I am creating a plugin that uses Lucene to index text fields and I >>>>> need to access *getConf()* and *getFilesystem()* of *HRegion, *the >>>>> Lucene indexes are split with the regions so I need " *HRegion >>>>> MyVar; ", *I am po

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread James Taylor
Shorter answer is "no". Your UDF may be executed on the client side as well (depending on the query) and there is of course no HRegion available from the client. On Tue, Apr 18, 2017 at 11:10 AM Sergey Soldatov wrote: > Well, theoretically there is a way of having a

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-14 Thread James Taylor
No, that's not possible. Thanks, James On Fri, Apr 14, 2017 at 9:48 AM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > If the UDF will be used only in select queries "SELECT my_udf(column) FROM > table WHERE id = 123" would it be possible to access the MyCoprocessor > instance on the

Re: Phoenix Query Server query logging

2017-04-12 Thread James Taylor
ing and what might cause > performance issues or errors during usage. > > Also, any other way to analyze these logs besides Splunk? > > On Tue, Apr 11, 2017 at 4:05 PM, James Taylor <jamestay...@apache.org> > wrote: > >> FWIW, we use our metric framework[1] to collect t

Re: Why does this work in MySQL but not Phoenix

2017-04-12 Thread James Taylor
The non standard @ syntax is not supported by Phoenix: (@rn:= if((@item = t.item_id) AND (@type = t.type),@rn + 1, if((@item:=t.item_id) AND (@type:=t.type),1,1) Also, I'd recommend breaking that query up into multiple queries to pinpoint any other non supported constructs. On

Re: Phoenix Query Server query logging

2017-04-11 Thread James Taylor
FWIW, we use our metric framework[1] to collect that information, outputting one log line per statement or query and then Splunk to look at it. [1] http://phoenix.apache.org/metrics.html On Tue, Apr 11, 2017 at 3:24 PM Michael Young wrote: > Yes, the tracing webapp is

Re: python-phoenixdb

2017-04-10 Thread James Taylor
gt;> Releasing is a different matter. I'm not sure how Phoenix releases are > >> currently handled, but for a Python library, it makes sense to upload > >> the package to PyPI since that's from where most Python developers are > >> getting packages from. Who w

Re: Ask performance advice

2017-04-04 Thread James Taylor
What other queries do you want to be fast? What version of Phoenix are you using? Have you seen our new Tuning Guide [1]? You could try moving OBJ_ID to end of PK and adding OBJ_GRP as the leading PK column (it's fine to include it in PK even if it's optional). Your index should be used, though.

Re: Can we simply update fat Java client without jar updates on workers?

2017-04-04 Thread James Taylor
Hi Yunus, No, unfortunately that won't work between minor releases. The server side needs to be upgraded (first) followed by the client side. See here [1] for more info. Thanks, James [1] http://phoenix.apache.org/upgrading.html On Tue, Apr 4, 2017 at 9:01 AM wrote:

Re: Get the Region and ID of the current row in a UDF

2017-03-28 Thread James Taylor
No. On Tue, Mar 28, 2017 at 10:19 AM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > So I cant get neither the name of the table, the name of the column, the > id of the row nor the region that "ptr" belongs to after calling: > > > > > > *Expression arg =

Re: Get the Region and ID of the current row in a UDF

2017-03-28 Thread James Taylor
No, that's not possible. Keep in mind that these expressions may be evaluated on the client or the server (it depends on the query). The could be evaluated on post aggregated rows too where the idea of which region no longer makes sense. On Sat, Mar 25, 2017 at 10:15 PM Cheyenne Forbes <

Re: Define parameters in queries

2017-03-28 Thread James Taylor
Phoenix supports JDBC, not any kind of scripting language like PLSQL or other variant. You can loook up online lots of examples of using PreparedStatement to bind variables in a query. On Tue, Mar 28, 2017 at 5:34 AM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > Can someone show an

Re: how can I cast PhoenixArray to ArrayList

2017-03-28 Thread James Taylor
racle API . There is getIntArray method to cast > Oracle Array to int[] .Phoenix array does not hava one > > 在 2017年3月27日,22:38,James Taylor <jamestay...@apache.org> 写道: > > The JDBC array is not an ArrayList. See > http://download.oracle.com/otn_hosted_doc/jdeveloper/905/jdbc-javadoc/oracle

Re: how can I cast PhoenixArray to ArrayList

2017-03-27 Thread James Taylor
The JDBC array is not an ArrayList. See http://download.oracle.com/otn_hosted_doc/jdeveloper/905/jdbc-javadoc/oracle/sql/ARRAY.html On Mon, Mar 27, 2017 at 3:50 AM 袁康(基础平台部) wrote: > When I query phoenix in java project > > occur:java.lang.ClassCastException: >

Re: Paged Queries Not Working

2017-03-24 Thread James Taylor
Bernard - the row value constructor syntax will work in the version you're using and it's a more scalable way of doing paging over HBase data. Thanks, James On Fri, Mar 24, 2017 at 3:16 AM, rajeshb...@apache.org < chrajeshbab...@gmail.com> wrote: > Offset is supported from Phoenix 4.8.0+

Re: python-phoenixdb

2017-03-23 Thread James Taylor
t; > part it worked. Definitely feel free to report any issues you run into: > > https://bitbucket.org/lalinsky/python-phoenixdb. It would be nice to > make > > this a "first-class citizen". > > > > James Taylor wrote: > > > >> Thanks, Dimitri. I c

[ANNOUNCE] Apache Phoenix 4.10 released

2017-03-23 Thread James Taylor
The Apache Phoenix team is pleased to announce the immediate availability of the 4.10.0 release. Apache Phoenix enables SQL-based OLTP and operational analytics for Hadoop using Apache HBase as its backing store and providing integration with other projects in the ecosystem such as Spark, Hive,

Re: Phoenix 4.9 on Hbase 1.3.0

2017-03-21 Thread James Taylor
The bug is HBASE-17587. Thanks, James On Tue, Mar 21, 2017 at 7:54 AM Mike Prendergast wrote: > According to the link below, it seems that Phoenix 4.9 won't work with > Hbase 1.3.0, and they are waiting for 1.3.1 for a bug fix. What is the > specific bug mentioned

Re: UNNEST is returning "Syntax error"

2017-03-19 Thread James Taylor
UNNEST is only supported in our calcite branch currently. There's been no release yet with support for it. If you're interested, I'd encourage you to get involved in the Phoenix/Calcite integration work or implement support for it in our 4.x branches. Thanks, James On Sun, Mar 19, 2017 at 4:59

Re: Differences between the date/time types

2017-03-17 Thread James Taylor
The DATE and TIME types in Phoenix are more or less the same. They store their data as an 8 byte long Epoch time and maintain millisecond granularity. The TIMESTAMP stores nanosecond data in a 12 bytes - an 8 byte long Epoch time plus 4 bytes for nanoseconds. Unless you need nanosecond

[ANNOUNCE] PhoenixCon 2017 is a go!

2017-03-15 Thread James Taylor
I'm excited to announce that the 2nd Annual Apache Phoenix conference, PhoenixCon 2017 will take place the day after HBaseCon in San Francisco on Tuesday, June 13th from 10:30am-6pm. For more details, including to RSVP and submit a talk proposal, click here:

Re: Phoenix ignoring index and index hint with some queries over mapped hbase tables.

2017-03-03 Thread James Taylor
Hi Marcin, There's a few things going on here: 1) An index created on a view over HBase data won't be maintained by Phoenix. You'd need to maintain it yourself through some other external means. If you create a table that maps to your HBase data, then it will be maintained. 2) An index would only

Re: Unable to connect to HBase using Phoenix JDBC Driver

2017-02-01 Thread James Taylor
Sounds like you're using the wrong jar on the server side. Or perhaps you're using a vendor distribution (in which case you'll need to use their supported Phoenix version). Please review the installation instructions as everything you need is in the one single jar. Thanks, James On Wed, Feb 1,

Re: Phoenix Implementation for OpenTSDB schema

2017-01-25 Thread James Taylor
e link provided. You linked the gmail sent mail > address i guess. Gmail can't redirect me to that and logs me back to my > inbox. Could you please repost it? > > Thanks. > > Regards. > > On Wed, Jan 25, 2017 at 9:44 PM, James Taylor <jamestay...@apache.org> > wrote:

Re: Phoenix Implementation for OpenTSDB schema

2017-01-25 Thread James Taylor
Please see this thread[1] for a description of how to do time series in Apache Phoenix. [1] https://mail.google.com/mail/u/0/#sent/15985640ed6c5e86 On Wed, Jan 25, 2017 at 4:26 AM, Dhawal Arora wrote: > Hi, > > I'm trying to map a clickstream data into Hbase Schema

Re: How can I set timeout option?

2017-01-23 Thread James Taylor
You can set the global phoenix.query.timeoutMs config in your client side hbase-site.xml [1], or set the same in the Connection Properties for it to apply just for that connection, or you can set it per query with the Statement.setQueryTimeout method:

Re: How to recreate table?

2017-01-18 Thread James Taylor
Yes, the consequences could be bad as you're bypassing the API that drops a table (which does more than just update the SYSTEM. CATALOG). Please see the recommendations in the previous emails. On Mon, Jan 16, 2017 at 10:45 PM Batyrshin Alexander <0x62...@gmail.com> wrote: > I've discovered that

Re: How to recreate table?

2017-01-16 Thread James Taylor
Couple of other options: 1) connect with a CURRENT_SCN [1] earlier than any timestamp for your data. For example, if you know you created the table a month ago, then use an SCN value of the epoch time for 1/1/2016. Then issue the DROP TABLE command from that connection and no data will be removed.

Re: Phoenix Secondary index question

2017-01-03 Thread James Taylor
Hi Udit, You'll need to use Phoenix APIs when you update your data if you want Phoenix to keep you index in sync with your table. Thanks, James On Tue, Jan 3, 2017 at 1:32 PM, Udit Mehta wrote: > Hi All, > > I am facing an issue while working with secondary indexes on a

Re: Zookeeper connection remains alive after DefaultStatisticsCollector's collecting stats done

2016-12-28 Thread James Taylor
This definitely sounds like a serious problem. Agreed, it should be closed immediately. Also, a minor compaction shouldn't trigger any stats collection logic. Please file a JIRA and we'll get this fixed in the next release. As always, patches are welcome. Thanks, James On Wed, Dec 28, 2016 at

Re: Shared Multi-Tenant Views

2016-12-27 Thread James Taylor
Hi Jeremy, Please file a JIRA for VIEW_INDEX_ID to be an int. The code would just need to be tolerant when reading the data if the length is two byte short versus four byte int. At write time, we'd just always write an int. If you could put together a patch that maintains backward compatibility,

Re: Can I reuse parameter values in phoenix query?

2016-12-08 Thread James Taylor
Yes - use :1 instead of ?1 On Thu, Dec 8, 2016 at 12:11 PM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > So instead of doing: > > query("select from table where c1 = ? or c2 = ?", [my_id, my_id]) > > i would do: > > query("select from table where c1 = ?1 or c2 = ?1",

Re: Secondary index breaks all queries

2016-12-07 Thread James Taylor
Thanks for narrowing down the cause, Karthick. Please file a JIRA with this information. James On Wed, Dec 7, 2016 at 12:45 PM, KARTHICK Duraisamy Soundararaj < ksoundara...@groupon.com> wrote: > I am able to create a secondary index on case-insensitive schema.table and > everything works

Re: phoenix upgrade issue

2016-12-06 Thread James Taylor
Hi Venkat, Did you ever update the SYSTEM.CATALOG table manually? If so, did you set the CURRENT_SCN property on the connection so that the timestamp of the modified rows continue to have the required timestamp (MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP_4_6_0 or 9 in this case)? It sounds like

Re: Inconsistent null behavior

2016-12-06 Thread James Taylor
Looks like a bug to me. If you can reproduce the issue outside of Python phoenixdb, using the Query Server directly, please file a JIRA. If not, it sounds like a Phython phoenixdb issue. Thanks, James On Tue, Dec 6, 2016 at 8:58 AM, Mark Heppner wrote: > I encountered

Re: Global Indexes and impact on availability

2016-12-05 Thread James Taylor
Have you tried local indexes? On Mon, Dec 5, 2016 at 2:35 PM Neelesh wrote: > Hello, > When a region server is under stress (hotspotting, or large replication, > call queue sizes hitting the limit, other processes competing with HBase > etc), we experience latency spikes

[ANNOUNCE] Apache Phoenix 4.9 released

2016-12-01 Thread James Taylor
Apache Phoenix enables OLTP and operational analytics for Apache Hadoop through SQL support using Apache HBase as its backing store and providing integration with other projects in the ecosystem such as Apache Spark, Apache Hive, Apache Pig, Apache Flume, and Apache MapReduce. We're pleased to

Re: Phoenix LIKE operator with bind parameters question/error

2016-11-28 Thread James Taylor
workin fine when using "=" and > it is not working with LIKE operator. > > It seems like parameters passed to LIKE operator are treated in a bit > different way. > > > > I am using DBeaver to test my queries and when I am using this notation > : (:1) or wirh paramet

Re: Unable to find cached index metadata

2016-11-26 Thread James Taylor
Not sure of the JIRA, but it sounds familiar. Try searching for it here: https://issues.apache.org/jira/browse/PHOENIX On Sat, Nov 26, 2016 at 1:22 PM Neelesh <neele...@gmail.com> wrote: > Thanks James! Is there a jira ref for the fix? > > On Nov 26, 2016 11:50 AM, "Jam

Re: Upsert works using /sqlline-thin.py and /sqlline.py but not in Zeppelin or other remote jdbc clients

2016-11-26 Thread James Taylor
in the connection or explicitly make the commit? > > On Sat, Nov 26, 2016 at 3:13 PM, James Taylor <jamestay...@apache.org> > wrote: > > Are you committing the data after the upsert (either by turning on auto > commit on the connection or doing an explicit commit)? > >

Re: Upsert works using /sqlline-thin.py and /sqlline.py but not in Zeppelin or other remote jdbc clients

2016-11-26 Thread James Taylor
Are you committing the data after the upsert (either by turning on auto commit on the connection or doing an explicit commit)? On Fri, Nov 25, 2016 at 12:20 PM Matt L wrote: > Hello All! > > Through Zeppelin I am trying to insert data into a table created by > Phoenix: > >

Re: Phoenix LIKE operator with bind parameters question/error

2016-11-18 Thread James Taylor
Use : instead, like this :1 Thanks, James On Fri, Nov 18, 2016 at 5:28 AM Bartłomiej Niemienionek < b.niemienio...@gmail.com> wrote: > Hi, > > > > I am trying to use Phoenix and I am facing some problems with LIKE > operator when used in the prepared statement with bind parameters. > > I don’t

Re: Phoenix-Spark plug in cannot select by column family name

2016-11-10 Thread James Taylor
gt; It works with no column family, but I expect that I do not need to make > sure column names are unique across different column families. > > > > Xindian > > > > > > *From:* James Taylor [mailto:jamestay...@apache.org > <jamestay...@apache.org>] > *Sent:*

Re: Phoenix-Spark plug in cannot select by column family name

2016-11-08 Thread James Taylor
Have you tried without the column family name? Unless the column names are not unique across all column families, you don't need to include the column family name. Thanks, James On Tue, Nov 8, 2016 at 2:19 PM, Long, Xindian wrote: > I have a table with multiple column

Re: PreparedStatement update counts seemingly incorrect

2016-11-03 Thread James Taylor
It sounds like PHOENIX-3178, but if not, please file a new JIRA with a way to reproduce it. Patches are most welcome too, of course. Thanks, James On Thu, Nov 3, 2016 at 4:38 PM, Julian Jaffe wrote: > When I execute an upsert with filters through the Phoenix JDBC

Re: python-phoenixdb 0.5 has been released

2016-11-03 Thread James Taylor
Awesome, Lukas! Thanks so much - this is great! On Thu, Nov 3, 2016 at 2:14 PM, Lukáš Lalinský wrote: > Hello, > > I have released new version of the Python client library for working with > Phoenix. Thanks to Mark Heppner it now uses the Protocol Buffers interface > instead

Re: Creating Covering index on Phoenix

2016-10-23 Thread James Taylor
CdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any

Re: Creating Covering index on Phoenix

2016-10-23 Thread James Taylor
Keep in mind that the CsvBulkLoadTool does not handle updating data in-place. It's expected that the data is unique by row and not updating existing data. If your data is write-once/append-only data, then you'll be ok, but otherwise you should stick with using the JDBC APIs. You're free to just

Re: How can I contribute clients?

2016-10-23 Thread James Taylor
That's awesome, Cheyenne! Is it packaged as an ODBC client or is the API different? If you send us a link to your Github repo and docs, we'll get links up on the website. Would be great to get an ODBC driver contributed to Phoenix and included in our distro. Thanks, James On Sunday, October 23,

Re: Index in Phoenix view on Hbase is not updated

2016-10-22 Thread James Taylor
Hi Mich, Phoenix indexes are only updated if you use Phoenix APIs to input the data. Thanks, James On Saturday, October 22, 2016, Ted Yu wrote: > The first statement creates index, not view. > > Can you check ? > > Cheers > > > On Oct 22, 2016, at 1:51 AM, Mich Talebzadeh

Re: Scanning big region parallely

2016-10-19 Thread James Taylor
Hi Sanooj, I'm not sure what you mean by "loading data in our HBase table into in-memory", but Phoenix queries tables in parallel, even within a region depending on how you've configured statistics and guideposts as described here: http://phoenix.apache.org/update_statistics.html Thanks, James

Re: Can phoenix support HBase's TimeStamp?

2016-10-16 Thread James Taylor
FYI, a couple of timestamp related features that Phoenix supports today include; - specify/filter on timestamp of a row: http://phoenix.apache.org/rowtimestamp.html - query as of a past timestamp:

Re: sc.phoenixTableAsRDD number of initial partitions

2016-10-14 Thread James Taylor
On Fri, Oct 14, 2016 at 12:37 AM, Antonio Murgia wrote: > We tried with an Upsert from select, but we ran into some memory issue > from the phoenix side. > > Do you have any suggestion to perform something like that? > You can try setting auto commit to true on the

Re: Can I change a String column's size and preserve the data?

2016-10-07 Thread James Taylor
Hi Zack, Phoenix doesn't support an ALTER TABLE statement that allows this, but you can modify the SYSTEM.CATALOG directly with an UPSERT VALUES call if you're careful. Make sure to: - Create a snapshot of the SYSTEM.CATALOG just in case something goes wrong - Ensure that other modifications

Re: Accessing phoenix tables in Spark 2

2016-10-07 Thread James Taylor
Hi Mich, I'd encourage you to use this mechanism mentioned by Josh: Another option is to use Phoenix-JDBC from within Spark Streaming. I've got a toy example of using Spark streaming with Phoenix DataFrames, but it could just as easily be a batched JDBC upsert. Trying to write directly to HBase

Re: Views and alter table

2016-10-07 Thread James Taylor
Hi Sanjooj, What version of Phoenix? Would you mind filing a JIRA with steps to reproduce the issue? Thanks, James On Friday, October 7, 2016, Sanooj Padmakumar wrote: > Hi All > > We get mutation state related error when we try altering a table to which > views are added.

Re: Creating view on a phoenix table throws Mismatched input error

2016-10-07 Thread James Taylor
FWIW, you can use an "alter view drop column" statement after you've created your view to remove columns, though. Thanks, James On Friday, October 7, 2016, Mich Talebzadeh wrote: > thanks Ankit > > Dr Mich Talebzadeh > > > > LinkedIn * >

Re: java.util.ConcurrentModificationException at java.util.HashMap while doing Upserts

2016-10-02 Thread James Taylor
ease let me know if there is any other better approach that i am missing. > > On Sun, Oct 2, 2016 at 3:50 PM, James Taylor <jamestay...@apache.org > <javascript:_e(%7B%7D,'cvml','jamestay...@apache.org');>> wrote: > >> Hi Anil, >> Make sure you're not sharing th

Re: java.util.ConcurrentModificationException at java.util.HashMap while doing Upserts

2016-10-02 Thread James Taylor
Hi Anil, Make sure you're not sharing the same Connection between multiple threads as it's not thread safe. Thanks, James On Sunday, October 2, 2016, anil gupta wrote: > Hi, > > We are running HDP2.3.4(HBase 1.1 and Phoenix 4.4). I have a MapReduce job > thats writing

<    1   2   3   4   5   6   7   >