Hello everybody,
For a few days I developed a MapReduce code to insert values in HBase with
Phoenix. But the code runs only in local and overcharge the machine.
Whatever changes I make I observe that the mapred.LocalJobRunner class is
systematically used.
Do you have an idea of the problem?
I attached to this post the execution logs of my program.
Thank you in advance for your help. :)
Feel free to ask me for more details if it can help.
Lucie Michaud
EOLAS, groupe Business & Decision<http://www.eolas.fr/> - tel.: +33 (0)4 76 44
50 50
Services en ligne managés 24/7 - e-commerce, e-administration, e-business
Suivez-nous sur Twitter : @BD_eolas<https://twitter.com/BD_eolas>
[http://www.eolas.fr/uploads/Image/52/504_680_Logo-Eolas-Bas-de-mail.gif]<http://www.eolas.fr/>
[adviz@tdr-adviz-cdh-w-01 src]$ hadoop jar testDistributedAggregation.jar
aggregation.retroactiveAggregation.RetroactiveAggregationApplication MEASURE
FIVEMIN
16/05/25 16:29:42 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x33fb6021 connecting to
ZooKeeperensemble=10.66.112.204:2181
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.5-cdh5.4.0--1, built on 04/21/2015 19:12 GMT
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:host.name=tdr-adviz-cdh-w-01.eolas-services.com
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.7.0_67
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Oracle Corporation
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/java/jdk1.7.0_67-cloudera/jre
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/etc/hadoop/conf:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.4.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/lib/hadoop/lib/native
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-573.8.1.el6.x86_64
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client environment:user.name=adviz
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:user.home=/home/adviz
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/home/adviz/lucie/modules_test_distributedAggregation/modules/src
16/05/25 16:29:42 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=10.66.112.204:2181
sessionTimeout=90000watcher=hconnection-0x33fb60210x0,
quorum=10.66.112.204:2181, baseZNode=/hbase
16/05/25 16:29:42 INFO zookeeper.ClientCnxn: Opening socket connection to
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181.Will not
attempt to authenticate using SASL (unknown error)
16/05/25 16:29:42 INFO zookeeper.ClientCnxn: Socket connection established to
tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181,initiating session
16/05/25 16:29:42 INFO zookeeper.ClientCnxn: Session establishment complete on
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181, sessionid =
0x453ec7535016c89, negotiated timeout = 90000
16/05/25 16:29:43 INFO metrics.Metrics: Initializing metrics system: phoenix
16/05/25 16:29:43 WARN impl.MetricsConfig: Cannot locate configuration: tried
hadoop-metrics2-phoenix.properties,hadoop-metrics2.properties
16/05/25 16:29:43 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10
second(s).
16/05/25 16:29:43 INFO impl.MetricsSystemImpl: phoenix metrics system started
16/05/25 16:29:44 INFO Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
16/05/25 16:29:44 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x2aa8b034 connecting to
ZooKeeperensemble=10.66.112.204:2181
16/05/25 16:29:44 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=10.66.112.204:2181
sessionTimeout=90000watcher=hconnection-0x2aa8b0340x0,
quorum=10.66.112.204:2181, baseZNode=/hbase
16/05/25 16:29:44 INFO zookeeper.ClientCnxn: Opening socket connection to
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181.Will not
attempt to authenticate using SASL (unknown error)
16/05/25 16:29:44 INFO zookeeper.ClientCnxn: Socket connection established to
tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181,initiating session
16/05/25 16:29:44 INFO zookeeper.ClientCnxn: Session establishment complete on
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181, sessionid =
0x453ec7535016c8a, negotiated timeout = 90000
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c8a
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c8a closed
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:45 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x3d4e158b connecting to
ZooKeeperensemble=10.66.112.204:2181
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=10.66.112.204:2181
sessionTimeout=90000watcher=hconnection-0x3d4e158b0x0,
quorum=10.66.112.204:2181, baseZNode=/hbase
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Opening socket connection to
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181.Will not
attempt to authenticate using SASL (unknown error)
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Socket connection established to
tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181,initiating session
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Session establishment complete on
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181, sessionid =
0x453ec7535016c8b, negotiated timeout = 90000
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c8b
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c8b closed
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:45 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x6aff6bd5 connecting to
ZooKeeperensemble=10.66.112.204:2181
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=10.66.112.204:2181
sessionTimeout=90000watcher=hconnection-0x6aff6bd50x0,
quorum=10.66.112.204:2181, baseZNode=/hbase
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Opening socket connection to
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181.Will not
attempt to authenticate using SASL (unknown error)
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Socket connection established to
tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181,initiating session
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Session establishment complete on
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181, sessionid =
0x453ec7535016c8c, negotiated timeout = 90000
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c8c
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c8c closed
16/05/25 16:29:45 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x4e42b9fd connecting to
ZooKeeperensemble=10.66.112.204:2181
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=10.66.112.204:2181
sessionTimeout=90000watcher=hconnection-0x4e42b9fd0x0,
quorum=10.66.112.204:2181, baseZNode=/hbase
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Opening socket connection to
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181.Will not
attempt to authenticate using SASL (unknown error)
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Socket connection established to
tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181,initiating session
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: Session establishment complete on
server tdr-adviz-cdh-w-01.eolas-services.com/10.66.112.204:2181, sessionid =
0x453ec7535016c8d, negotiated timeout = 90000
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:45 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c8d
16/05/25 16:29:45 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c8d closed
16/05/25 16:29:45 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:50 INFO Configuration.deprecation: session.id is deprecated.
Instead, use dfs.metrics.session-id
16/05/25 16:29:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
16/05/25 16:29:50 WARN mapred.JobClient: Use GenericOptionsParser for parsing
the arguments. Applications should implement Tool for the same.
16/05/25 16:29:50 INFO Configuration.deprecation: io.bytes.per.checksum is
deprecated. Instead, use dfs.bytes-per-checksum
16/05/25 16:29:50 INFO Configuration.deprecation: io.bytes.per.checksum is
deprecated. Instead, use dfs.bytes-per-checksum
16/05/25 16:29:50 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x4ed8d66f connecting to ZooKeeperensemble=localhost:2181
16/05/25 16:29:50 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181
sessionTimeout=90000watcher=hconnection-0x4ed8d66f0x0, quorum=localhost:2181,
baseZNode=/hbase
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/127.0.0.1:2181, sessionid =0x453ec7535016c8f, negotiated
timeout = 90000
16/05/25 16:29:50 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x506c1818 connecting to ZooKeeperensemble=localhost:2181
16/05/25 16:29:50 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181
sessionTimeout=90000watcher=hconnection-0x506c18180x0, quorum=localhost:2181,
baseZNode=/hbase
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/127.0.0.1:2181, sessionid =0x453ec7535016c90, negotiated
timeout = 90000
16/05/25 16:29:50 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:50 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c90
16/05/25 16:29:50 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c90 closed
16/05/25 16:29:50 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:51 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x77b18bb3 connecting to ZooKeeperensemble=localhost:2181
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181
sessionTimeout=90000watcher=hconnection-0x77b18bb30x0, quorum=localhost:2181,
baseZNode=/hbase
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/127.0.0.1:2181, sessionid =0x453ec7535016c91, negotiated
timeout = 90000
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c91
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c91 closed
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:51 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x57d5d80f connecting to ZooKeeperensemble=localhost:2181
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181
sessionTimeout=90000watcher=hconnection-0x57d5d80f0x0, quorum=localhost:2181,
baseZNode=/hbase
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/127.0.0.1:2181, sessionid =0x453ec7535016c92, negotiated
timeout = 90000
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c92
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c92 closed
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:51 INFO zookeeper.RecoverableZooKeeper: Process
identifier=hconnection-0x3b30baff connecting to ZooKeeperensemble=localhost:2181
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=localhost:2181
sessionTimeout=90000watcher=hconnection-0x3b30baff0x0, quorum=localhost:2181,
baseZNode=/hbase
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Opening socket connection to
server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
(unknown error)
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Socket connection established to
localhost/127.0.0.1:2181, initiating session
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: Session establishment complete on
server localhost/127.0.0.1:2181, sessionid =0x453ec7535016c93, negotiated
timeout = 90000
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing master protocol: MasterService
16/05/25 16:29:51 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c93
16/05/25 16:29:51 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c93 closed
16/05/25 16:29:51 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:52 INFO mapred.JobClient: Running job: job_local573864567_0001
16/05/25 16:29:52 INFO mapred.LocalJobRunner: OutputCommitter set in config null
16/05/25 16:29:52 INFO mapred.LocalJobRunner: OutputCommitter is
org.apache.phoenix.mapreduce.PhoenixOutputCommitter
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Waiting for map tasks
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Starting task:
attempt_local573864567_0001_m_000000_0
16/05/25 16:29:52 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
16/05/25 16:29:52 INFO util.ProcessTree: setsid exited with exit code 0
16/05/25 16:29:52 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1309712b
16/05/25 16:29:52 INFO mapred.MapTask: Processing split:
org.apache.phoenix.mapreduce.PhoenixInputSplit@b4eba526
16/05/25 16:29:52 INFO mapred.MapTask: Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
16/05/25 16:29:52 INFO mapred.MapTask: io.sort.mb = 100
16/05/25 16:29:52 INFO mapred.MapTask: data buffer = 79691776/99614720
16/05/25 16:29:52 INFO mapred.MapTask: record buffer = 262144/327680
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.MapTask: Starting flush of map output
16/05/25 16:29:52 INFO mapred.MapTask: Finished spill 0
16/05/25 16:29:52 INFO mapred.Task: Task:attempt_local573864567_0001_m_000000_0
is done. And is in the process of commiting
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task attempt_local573864567_0001_m_000000_0
is allowed to commit now
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task
'attempt_local573864567_0001_m_000000_0' done.
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Finishing task:
attempt_local573864567_0001_m_000000_0
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Starting task:
attempt_local573864567_0001_m_000001_0
16/05/25 16:29:52 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
16/05/25 16:29:52 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7e6abaff
16/05/25 16:29:52 INFO mapred.MapTask: Processing split:
org.apache.phoenix.mapreduce.PhoenixInputSplit@a313a8d1
16/05/25 16:29:52 INFO mapred.MapTask: Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
16/05/25 16:29:52 INFO mapred.MapTask: io.sort.mb = 100
16/05/25 16:29:52 INFO mapred.MapTask: data buffer = 79691776/99614720
16/05/25 16:29:52 INFO mapred.MapTask: record buffer = 262144/327680
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.MapTask: Starting flush of map output
16/05/25 16:29:52 INFO mapred.MapTask: Finished spill 0
16/05/25 16:29:52 INFO mapred.Task: Task:attempt_local573864567_0001_m_000001_0
is done. And is in the process of commiting
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task attempt_local573864567_0001_m_000001_0
is allowed to commit now
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task
'attempt_local573864567_0001_m_000001_0' done.
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Finishing task:
attempt_local573864567_0001_m_000001_0
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Starting task:
attempt_local573864567_0001_m_000002_0
16/05/25 16:29:52 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
16/05/25 16:29:52 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4af8c887
16/05/25 16:29:52 INFO mapred.MapTask: Processing split:
org.apache.phoenix.mapreduce.PhoenixInputSplit@b3945ee
16/05/25 16:29:52 INFO mapred.MapTask: Map output collector class =
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
16/05/25 16:29:52 INFO mapred.MapTask: io.sort.mb = 100
16/05/25 16:29:52 INFO mapred.MapTask: data buffer = 79691776/99614720
16/05/25 16:29:52 INFO mapred.MapTask: record buffer = 262144/327680
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.MapTask: Starting flush of map output
16/05/25 16:29:52 INFO mapred.MapTask: Finished spill 0
16/05/25 16:29:52 INFO mapred.Task: Task:attempt_local573864567_0001_m_000002_0
is done. And is in the process of commiting
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task attempt_local573864567_0001_m_000002_0
is allowed to commit now
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Task: Task
'attempt_local573864567_0001_m_000002_0' done.
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Finishing task:
attempt_local573864567_0001_m_000002_0
16/05/25 16:29:52 INFO mapred.LocalJobRunner: Map task executor complete.
16/05/25 16:29:52 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
16/05/25 16:29:52 INFO mapred.Task: Using ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@217b10dd
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapred.Merger: Merging 3 sorted segments
16/05/25 16:29:52 INFO mapred.Merger: Down to the last merge-pass, with 3
segments left of total size: 18192 bytes
16/05/25 16:29:52 INFO mapred.LocalJobRunner:
16/05/25 16:29:52 INFO mapreduce.PhoenixInputFormat: UseUpsertColumns=true,
upsertColumnList.size()=9,
upsertColumnList=id,startTimestamp,timestamp,first,last,min,max,avg,nb
16/05/25 16:29:52 INFO mapreduce.PhoenixInputFormat: Phoenix Custom Upsert
Statement: UPSERT INTO "mea_aggr_5minV2" ("id","startTimestamp",
"0"."timestamp", "0"."first", "0"."last", "0"."min", "0"."max", "0"."avg",
"0"."nb") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
16/05/25 16:29:57 INFO mapred.Task: Task:attempt_local573864567_0001_r_000000_0
is done. And is in the process of commiting
16/05/25 16:29:57 INFO mapred.LocalJobRunner:
16/05/25 16:29:57 INFO mapred.Task: Task attempt_local573864567_0001_r_000000_0
is allowed to commit now
16/05/25 16:29:57 INFO mapred.LocalJobRunner: reduce > reduce
16/05/25 16:29:57 INFO mapred.Task: Task
'attempt_local573864567_0001_r_000000_0' done.
16/05/25 16:29:58 INFO mapred.JobClient: map 0% reduce 100%
16/05/25 16:29:58 INFO mapred.JobClient: Job complete: job_local573864567_0001
16/05/25 16:29:58 INFO mapred.JobClient: Counters: 25
16/05/25 16:29:58 INFO mapred.JobClient: File System Counters
16/05/25 16:29:58 INFO mapred.JobClient: FILE: Number of bytes read=222451
16/05/25 16:29:58 INFO mapred.JobClient: FILE: Number of bytes
written=1002250
16/05/25 16:29:58 INFO mapred.JobClient: FILE: Number of read operations=0
16/05/25 16:29:58 INFO mapred.JobClient: FILE: Number of large read
operations=0
16/05/25 16:29:58 INFO mapred.JobClient: FILE: Number of write operations=0
16/05/25 16:29:58 INFO mapred.JobClient: HDFS: Number of bytes read=0
16/05/25 16:29:58 INFO mapred.JobClient: HDFS: Number of bytes written=0
16/05/25 16:29:58 INFO mapred.JobClient: HDFS: Number of read operations=0
16/05/25 16:29:58 INFO mapred.JobClient: HDFS: Number of large read
operations=0
16/05/25 16:29:58 INFO mapred.JobClient: HDFS: Number of write operations=0
16/05/25 16:29:58 INFO mapred.JobClient: Map-Reduce Framework
16/05/25 16:29:58 INFO mapred.JobClient: Map input records=433
16/05/25 16:29:58 INFO mapred.JobClient: Map output records=433
16/05/25 16:29:58 INFO mapred.JobClient: Map output bytes=17320
16/05/25 16:29:58 INFO mapred.JobClient: Input split bytes=2024
16/05/25 16:29:58 INFO mapred.JobClient: Combine input records=0
16/05/25 16:29:58 INFO mapred.JobClient: Combine output records=0
16/05/25 16:29:58 INFO mapred.JobClient: Reduce input groups=40
16/05/25 16:29:58 INFO mapred.JobClient: Reduce shuffle bytes=0
16/05/25 16:29:58 INFO mapred.JobClient: Reduce input records=433
16/05/25 16:29:58 INFO mapred.JobClient: Reduce output records=58
16/05/25 16:29:58 INFO mapred.JobClient: Spilled Records=866
16/05/25 16:29:58 INFO mapred.JobClient: CPU time spent (ms)=0
16/05/25 16:29:58 INFO mapred.JobClient: Physical memory (bytes) snapshot=0
16/05/25 16:29:58 INFO mapred.JobClient: Virtual memory (bytes) snapshot=0
16/05/25 16:29:58 INFO mapred.JobClient: Total committed heap usage
(bytes)=1679818752
16/05/25 16:29:58 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c89
16/05/25 16:29:58 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c89 closed
16/05/25 16:29:58 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:58 INFO client.ConnectionManager$HConnectionImplementation:
Closing zookeeper sessionid=0x453ec7535016c8f
16/05/25 16:29:58 INFO zookeeper.ClientCnxn: EventThread shut down
16/05/25 16:29:58 INFO zookeeper.ZooKeeper: Session: 0x453ec7535016c8f closed
