Liwei Lin created PARQUET-529:
---------------------------------
Summary: Avoid evoking job.toString() in ParquetLoader
Key: PARQUET-529
URL: https://issues.apache.org/jira/browse/PARQUET-529
Project: Parquet
Issue Type: Bug
Components: parquet-pig
Affects Versions: 1.8.0, 1.8.1
Reporter: Liwei Lin
Assignee: Liwei Lin
Fix For: 1.9.0
When ran under hadoop2 environment and log level setting to _DEBUG_,
_ParquetLoader_ would evoke _job.toString()_ in several methods, which might
cause the whole application to stop due to :
{quote}
java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
at java.lang.String.valueOf(String.java:2847)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at org.apache.parquet.pig.ParquetLoader.getSchema(ParquetLoader.java:260)
at
org.apache.parquet.pig.TestParquetLoader.testSchema(TestParquetLoader.java:54)
...
{quote}
The reason is that in the hadoop 2.x branch,
_org.apache.hadoop.mapreduce.Job.toString()_ has added an
_ensureState(JobState.RUNNING)_ check; see map-reduce: Job.java#452. In
contrast, the hadoop 1.x branch does not contain such checks, so
_ParquetLoader_ works well.
This ticket simply avoids evoking _job.toString()_ in _ParquetLoader_.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
