Re: Phoenix Input Format for Map Reduce
Thanks Ravi for providing required details. Gagan On Fri, Nov 14, 2014 at 1:24 PM, Ravi Kiran maghamraviki...@gmail.com wrote: Hi Gagan, The support for MR is a WIP. We are working on it and the first draft of the code is at https://github.com/apache/phoenix/pull/23 . A preview of the API is available at https://gist.github.com/mravi/501fa00b942764eb0dca. I have also created a new JIRA ticket https://issues.apache.org/jira/browse/PHOENIX-1454 for this. Regarding using the PhoenixInputFormat of the phoenix-pig module, you will need to manually set few configuration parameters like the table name, columns etc. Apparently, the integration of MR with the PhoenixInputFormat of pig-module isn't a smooth one without lot of customizations. Regards Ravi On Thu, Nov 13, 2014 at 11:18 PM, Gagan Agrawal agrawalga...@gmail.com wrote: Hi, I was looking for Phoenix Input Format to read from phoenix powered hbase in my Map Reduce jobs. Came across following project which provides pig implementation of Phonenix Input Format. https://github.com/apache/phoenix/tree/master/phoenix-pig/src/main/java/org/apache/phoenix/pig/hadoop I wanted to understand if there is anything specific to pig in this? Or can we use it in Map Reduce also? Will I need to do any customization to get it work with MR? Is this stable and has been used in production? Can I use it with latest phoenix version or only specific version is supported. Thanks and Regards, Gagan Agrawal
Re: Phoenix Input Format for Map Reduce
Yes Thanks! Up until now we just used the standard Input formats and used the type converters inside Phoenix core :-) Sent from mobile device Am 14.11.2014 um 09:13 schrieb Gagan Agrawal agrawalga...@gmail.com: Thanks Ravi for providing required details. Gagan On Fri, Nov 14, 2014 at 1:24 PM, Ravi Kiran maghamraviki...@gmail.com wrote: Hi Gagan, The support for MR is a WIP. We are working on it and the first draft of the code is at https://github.com/apache/phoenix/pull/23 . A preview of the API is available at https://gist.github.com/mravi/501fa00b942764eb0dca. I have also created a new JIRA ticket https://issues.apache.org/jira/browse/PHOENIX-1454 for this. Regarding using the PhoenixInputFormat of the phoenix-pig module, you will need to manually set few configuration parameters like the table name, columns etc. Apparently, the integration of MR with the PhoenixInputFormat of pig-module isn't a smooth one without lot of customizations. Regards Ravi On Thu, Nov 13, 2014 at 11:18 PM, Gagan Agrawal agrawalga...@gmail.com wrote: Hi, I was looking for Phoenix Input Format to read from phoenix powered hbase in my Map Reduce jobs. Came across following project which provides pig implementation of Phonenix Input Format. https://github.com/apache/phoenix/tree/master/phoenix-pig/src/main/java/org/apache/phoenix/pig/hadoop I wanted to understand if there is anything specific to pig in this? Or can we use it in Map Reduce also? Will I need to do any customization to get it work with MR? Is this stable and has been used in production? Can I use it with latest phoenix version or only specific version is supported. Thanks and Regards, Gagan Agrawal
phoenix backup and recovery
Is it possible to use hbase snapshots http://hbase.apache.org/book/ops.snapshots.html with phoenix? and what is the recommended process for backup and recovery thanks Alex
Re: phoenix backup and recovery
You absolutely can use snapshots with phoenix. You would need to snapshot both the phoenix metadata table and the table you want to snapshot. Then on restore, you restore both those tables to new tables, point phoenix there and get the data you need. Missing pieces: 1) I'm not sure there is a way to just copy over the rows you would need from the metadata to bulk move the restored table back. This would be a great contribution. 2) Schema changes need to be locked while doing the snapshot so you dont see the wrong schema by accident. This may or may not exist - check the docs. What you really want is a per-table schema lock, but a wholesale schema lock would be sufficient, though slightly less clean. On Fri, Nov 14, 2014, 8:59 AM Alex Kamil alex.ka...@gmail.com wrote: Is it possible to use hbase snapshots http://hbase.apache.org/book/ops.snapshots.html with phoenix? and what is the recommended process for backup and recovery thanks Alex
