weird result I got when I try row_timestamp feature

2015-12-11 Thread Roc Chu
Hi all I try to use phoenix 4.6 to deal with time series data, this is my table schema create table log_data_46201512( server_timestamp timestamp not null, app_id varchar, client_timestamp timestamp, client_ip INTEGER not null, cluster_id varchar, host_id varchar, device_uuid varchar, url

RE: CsvBulkUpload not working after upgrade to 4.6

2015-12-11 Thread Riesland, Zack
Thanks James, I did. Unfortunately, Hortonworks only supports software that they provide in their stack, and they only provide 4.2.2 for our version of the HDP stack. Even if we upgrade to the latest HDP, we would only get support for Phoenix 4.4 FWIW, I am also unable to connect Aqua Data

Re: Copy table between hbase clusters

2015-12-11 Thread James Taylor
Your analysis of the row key structure is correct. Those are all fixed types (4 + 4 + 8 +8 + 2 = 26 bytes for the key). If you're going from 0.94 to 0.98, there's stuff you need to do to get your data into the new format. Best to ask about this on the HBase user list or look it up in the

Re: weird result I got when I try row_timestamp feature

2015-12-11 Thread Roc Chu
Thanks for your reply. In my test table, I used the row_timestamp feature and I lost ROW_TIMESTAMP key word when I copied schema to this mail, when I finished data load, I executed that query several times, and got different cont, it seemed that the result affected by where the data is, in hbase

Fwd: Phoenix table stats query performance issue

2015-12-11 Thread Li Gao
Hi Phoenix community, We are encountering general performance degradation when doing table stats query over the leading primary key column in a Phoenix table. The table schema, used query, JMeter results observed over 1-hour window, and the python script that slowly generate data into the table.

Re: Phoenix table stats query performance issue

2015-12-11 Thread James Taylor
Hi Li, That's not performance degradation. Your query requires a full table scan to calculate the row count. It's going to get slower as the table grows. Your new queries will remain fast as the table grows in size. Thanks, James On Fri, Dec 11, 2015 at 7:46 PM, Li Gao

Memory issues in webapp with HBase integration using phoenix

2015-12-11 Thread Vijay Vangapandu
Hi Guys, We recently deployed an application which serves one of our online pages traffic. Our web application is jetty based and the virtual machine configuration listed below. We allocated 3.5GB to heap but java is taking about 5.5GB on startup and in 20hours it’s going all the way to

Mapping Hive Table To Phoenix Table

2015-12-11 Thread Justin Workman
We are trying to map a Hive table to a Phoenix table and having problems when we use a composite key made from several columns in the table. It appears that these columns are not exposed independently when trying to map to Hive because they are not actual columns in the underlying HBase table. Is

Index rows are not updated when the index key updated using bulk loader

2015-12-11 Thread Afshin Moazami
Hi, I guess it is a bug, but before posting in Jira, I want to see if anyone has an idea here. I found out the map reduce csv bulk load tool doesn't behave the same as UPSERTs. Is it by design or a bug? Here is the queries for creating table and index: CREATE TABLE mySchema.mainTable ( id

Re: weird result I got when I try row_timestamp feature

2015-12-11 Thread Samarth Jain
Hi Roc, FWIW, looking at your schema, it doesn't look like you are using the ROW_TIMESTAMP feature. The constraint part of your DDL needs to be changed like this: CONSTRAINT my_pk PRIMARY KEY ( server_timestamp ROW_TIMESTAMP, app_id, client_ip, cluster_id,host_id,api ) For the issue of getting

Re: Mapping Hive Table To Phoenix Table

2015-12-11 Thread James Taylor
There's some work that was done to make Phoenix and Hive work well together here: https://github.com/apache/phoenix/pull/74 and here: https://github.com/nmaillard/Phoenix-Hive The pull is out of date, but could likely be revived - just needs an owner and needs to be brought over the finish line.

Re: Phoenix table stats query performance issue

2015-12-11 Thread Li Gao
Per a separate thread with suggestions from James, we modified the queries to split into separate individual queries and the response time seems improving. select id from pk.tests order by id limit 1; -- query for min id select id from pk.tests order by id desc limit 1; -- query for max id The