Just to close the loop on this...
Did not have time to experiment with other EMR versions, so just going with
emr-4.9.2 for the near future since Pig Phoenix storage works as expected
when running the script from the command line.
However, made an action item for a future date to try submitting
Thanks for the extra info! Will let everyone know if I solve this.
On Mon, Aug 21, 2017 at 4:24 PM, anil gupta wrote:
> And forgot to mention that we invoke our pig scripts through oozie.
>
> On Mon, Aug 21, 2017 at 2:20 PM, anil gupta wrote:
>
>>
And forgot to mention that we invoke our pig scripts through oozie.
On Mon, Aug 21, 2017 at 2:20 PM, anil gupta wrote:
> Sorry, cant share the pig script.
> Here is what we are registering:
> REGISTER /usr/lib/phoenix/phoenix-4.7.0-HBase-1.2-client.jar;
> REGISTER
Sorry, cant share the pig script.
Here is what we are registering:
REGISTER /usr/lib/phoenix/phoenix-4.7.0-HBase-1.2-client.jar;
REGISTER /usr/lib/pig/lib/piggybank.jar;
Following is the classpath of Hadoop and Yarn:
[hadoop@ip-52-143 ~]$ hadoop classpath
Hmm... just repeated my test on emr-5.2.0. This time I went with the
default EMR console selections for master and core nodes (2 of them).
When running my simple Pig Phoenix store script, still getting the errors I
got for other 5.x.x versions:
2017-08-21 17:50:52,431 [ERROR] [main]
Anil,
That's good news (about 5.2). Any chance I could see your
- full pig command line
- PIG_CLASSPATH env variable
- pig script or at least the REGISTER and PhoenixHBaseStorage() lines?
Might help me figure out what I'm doing wrong or differently.
One thing I did not mention
Hey Steve,
We are currently using EMR5.2 and pig-phoenix is working fine for us. We
are gonna try EMR5.8 next week.
HTH,
Anil
On Fri, Aug 18, 2017 at 9:00 AM, Steve Terrell
wrote:
> More info...
>
> By trial and error, I tested different EMR versions and made a little
>
More info...
By trial and error, I tested different EMR versions and made a little
incomplete list of which ones support Pig Phoenix storage and which ones
don't:
emr-5.8.0 JacksonJaxbJsonProvider error
emr-5.6.0 JacksonJaxbJsonProvider error
emr-5.4.0 JacksonJaxbJsonProvider error
emr-5.3.1
I'm running EMR 5.8.0 with these applications installed:
Pig 0.16.0, Phoenix 4.11.0, HBase 1.3.1
Here is my pig script (try.pig):
REGISTER /usr/lib/phoenix/phoenix-4.11.0-HBase-1.3-client.jar;
A = load '/steve/a.txt' as (TXT:chararray);
store A into 'hbase://A_TABLE' using