Sambit Sahoo created PIG-3272:
---------------------------------
Summary: While running a pig job i am seeing some delay
Key: PIG-3272
URL: https://issues.apache.org/jira/browse/PIG-3272
Project: Pig
Issue Type: Bug
Components: data
Affects Versions: 0.10.1
Reporter: Sambit Sahoo
2013-04-09 06:29:42,639 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=MAP, sessionId=
2013-04-09 06:29:43,947 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 1024
2013-04-09 06:29:44,434 INFO org.apache.hadoop.mapred.MapTask: data buffer =
816043776/1020054736
2013-04-09 06:29:44,435 INFO org.apache.hadoop.mapred.MapTask: record buffer =
2684354/3355443
2013-04-09 06:29:44,448 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy:
Snappy native library is available
2013-04-09 06:29:44,448 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy:
Snappy native library loaded
2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: Spilling map
output: record full = true
2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0;
bufend = 423372603; bufvoid = 1020054736
2013-04-09 06:48:29,783 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0;
kvend = 2684354; length = 3355443
2013-04-09 06:49:32,969 INFO org.apache.hadoop.io.compress.CodecPool: Got
brand-new compressor
In the above log you will see, after Snappy native library loaded it took
around 20mins to proceed.
Can anyone tell me the issue and the resolution?
is it due to some network issue?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
