[
https://issues.apache.org/jira/browse/ORC-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thrinath Dosapati updated ORC-547:
----------------------------------
Issue Type: Bug (was: Test)
> ORC write on Map Reduce fwk is extremely slow
> ---------------------------------------------
>
> Key: ORC-547
> URL: https://issues.apache.org/jira/browse/ORC-547
> Project: ORC
> Issue Type: Bug
> Components: MapReduce
> Affects Versions: 1.3.3
> Environment: Map Reduce FWK
> Reporter: Thrinath Dosapati
> Priority: Major
> Attachments: orc_slow_write_log.txt, sample_record.json
>
>
> Recently, we have encountered cases where the ORC write is extremely slow for
> certain workloads.
> I tried to run this on Spark env, but the behaviour remains same
> What could be the reason for the slowness?
> Schema :
> struct<rc:struct<cc:struct<appv:string,cht:string>,pc,ac:array<struct<layer:string,abid:string>>,mp:string,rsc:bigint,pt:string,ai:struct<supercat:string,subcat:string,v:string,cat:string>,prid:string,pid:array<string>,rid:string,uc:struct<abid:string,aid:string>,p:array<struct<productid:string,meta:array<struct<mv:string,mk:string>>,nid:string,lid:string>>,sc:array<struct<score:double,sid:string>>,ui:struct<ss:string,dg:struct<mds:string,fds:string>,ps:string,bg:struct<ms:string,fs:string>,ms:string,ul:array<struct<c:string,s:string,p:string>>,iscc:boolean,ic:boolean,rfmb:struct<rb:string,fb:string,mb:string,rfmsg:string,imlb:boolean>>,pck:string,pi:string,dc:struct<os:string,ip:string,did:string>>,rws:array<struct<rccs:array<struct<eid:string,bc:string,mp:string,lid:string,nid:string,cm:array<struct<mv:string,mk:string>>,mtomlfs:array<struct<rv:string,lid:string,ms:string,mid:string,mv:string,mlfs:array<struct<fw:string,f:string>>>>,rpid:string,et:string,dt:string,cs:string,ct:string,t:string,cid:string>>,wm:array<struct<mv:string,mk:string>>,mtomlfs:array<struct<rv:string,lid:string,ms:string,mid:string,mv:string,mlfs:array<struct<fw:string,f:string>>>>,wc:struct<murl:string,rt:string,djct:string,wimpid:string,va:string,title:string,ws:string,wc:string,mtext:string,wt:string,vt:string,urms:array<struct<rk:string,dc:bigint>>,mrcc:bigint,sc:bigint>>>>
>
> Logs and sample records are attached for reference.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
