[
https://issues.apache.org/jira/browse/PHOENIX-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-2083:
----------------------------------
Summary: Pig maps splits are very uneven (was: Pig maps splits are very
unevent)
> Pig maps splits are very uneven
> -------------------------------
>
> Key: PHOENIX-2083
> URL: https://issues.apache.org/jira/browse/PHOENIX-2083
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: Brian Johnson
> Labels: verify
> Fix For: 4.8.0
>
>
> When running a pig job on MR with the Phoenix loader we got about 75 maps
> tasks, but there was huge amount of skew in how the records were allocated
> and the vast majority of them went to about 20 mappers and 5 got nothing at
> all.
> Task
> Value
> task_1433431098673_66646_m_000042 0
> task_1433431098673_66646_m_000057 0
> task_1433431098673_66646_m_000061 0
> task_1433431098673_66646_m_000067 0
> task_1433431098673_66646_r_000000 0
> task_1433431098673_66646_m_000031 127242
> task_1433431098673_66646_m_000026 130669
> task_1433431098673_66646_m_000017 179685
> task_1433431098673_66646_m_000068 190741
> task_1433431098673_66646_m_000040 191062
> task_1433431098673_66646_m_000056 191509
> task_1433431098673_66646_m_000053 191518
> task_1433431098673_66646_m_000060 191560
> task_1433431098673_66646_m_000048 191579
> task_1433431098673_66646_m_000041 191623
> task_1433431098673_66646_m_000047 191686
> task_1433431098673_66646_m_000065 191720
> task_1433431098673_66646_m_000064 191726
> task_1433431098673_66646_m_000054 191763
> task_1433431098673_66646_m_000066 191871
> task_1433431098673_66646_m_000052 191875
> task_1433431098673_66646_m_000045 191908
> task_1433431098673_66646_m_000049 191914
> task_1433431098673_66646_m_000063 192124
> task_1433431098673_66646_m_000058 192352
> task_1433431098673_66646_m_000069 192352
> task_1433431098673_66646_m_000044 192519
> task_1433431098673_66646_m_000007 529769
> task_1433431098673_66646_m_000018 584940
> task_1433431098673_66646_m_000005 585864
> task_1433431098673_66646_m_000003 697683
> task_1433431098673_66646_m_000016 709321
> task_1433431098673_66646_m_000008 710190
> task_1433431098673_66646_m_000004 710774
> task_1433431098673_66646_m_000011 711818
> task_1433431098673_66646_m_000038 713862
> task_1433431098673_66646_m_000037 714577
> task_1433431098673_66646_m_000022 716796
> task_1433431098673_66646_m_000014 717478
> task_1433431098673_66646_m_000025 722809
> task_1433431098673_66646_m_000030 723182
> task_1433431098673_66646_m_000024 723378
> task_1433431098673_66646_m_000013 731836
> task_1433431098673_66646_m_000010 732525
> task_1433431098673_66646_m_000001 734611
> task_1433431098673_66646_m_000036 739874
> task_1433431098673_66646_m_000072 1810925
> task_1433431098673_66646_m_000039 1923212
> task_1433431098673_66646_m_000059 2014210
> task_1433431098673_66646_m_000055 2287499
> task_1433431098673_66646_m_000074 2887750
> task_1433431098673_66646_m_000073 3049942
> task_1433431098673_66646_m_000029 3156535
> task_1433431098673_66646_m_000071 3841375
> task_1433431098673_66646_m_000027 4001882
> task_1433431098673_66646_m_000051 4343619
> task_1433431098673_66646_m_000034 5363718
> task_1433431098673_66646_m_000050 7734798
> task_1433431098673_66646_m_000020 9543930
> task_1433431098673_66646_m_000070 10058382
> task_1433431098673_66646_m_000046 10143291
> task_1433431098673_66646_m_000062 10263757
> task_1433431098673_66646_m_000032 10908072
> task_1433431098673_66646_m_000015 11182800
> task_1433431098673_66646_m_000000 11300385
> task_1433431098673_66646_m_000043 11359327
> task_1433431098673_66646_m_000021 12632598
> task_1433431098673_66646_m_000009 14598258
> task_1433431098673_66646_m_000028 14698359
> task_1433431098673_66646_m_000033 16407474
> task_1433431098673_66646_m_000012 17944269
> task_1433431098673_66646_m_000023 20568188
> task_1433431098673_66646_m_000035 21656353
> task_1433431098673_66646_m_000002 27413291
> task_1433431098673_66646_m_000006 35573698
> task_1433431098673_66646_m_000019 35717128
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
