[jira] [Commented] (HIVE-9067) OrcFileMergeOperator may create merge file that does not match properties of input files

[Feng Yuan (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HIVE-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934452#comment-14934452
 ] 

Feng Yuan commented on HIVE-9067:
---------------------------------

like this:
test_table`s data:
img
top
\N
\N
\N
0
0
mobile_contentgeneral
66
na
Minix
exact_match
Android
false
false
true
NEO-X7-216A
0
216
Mozilla/5.0
 (Linux; U; Android 4.2.2; ar-eg; NEO-X7-216A Build/JDQ39) AppleWebKit/534.30 
(KHTML, like Gecko) Version/4.0 Safari/534.30
none
\N
Online Breedband 
B.V.
eindhoven

NL
83.117.26.125

true
true
51.44
5.48
\N
5654
06
0
14182415864288907390719148304469
json
Live
122
\N
1418241586428
8907390719148304469
mobapp
0
0||
386
64.14.181.220

17597571-80a7-11e4-b1d6-52540004
1418169600
1418238000
img
top
\N
65772487
\N
0
0
politicsmobile_contentdatinggeneralringtones
66
na
Microsoft
exact_match
Windows
false
false
false
Generic
 Windows Mobile
320
240
Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; 
PPC)
none
\N
Aircel Ltd.
kolkata
mobile wireless
IN
101.218.108.20
mobile 
gateway
true
true
22.5697021484375
88.36968994140625
DISHNETIND
\N
28
0
14182415864284741441834639756033
xml
Live
76
\N
1418241586428
4741441834639756033
mobweb
0
0||
60
54.175.7.64
\N
\N
1418169600
1418238000
img
top
\N
\N
\N
0
0
mobile_contentgeneral
66
na
Minix
exact_match
Android
false
false
true
NEO-X7-216A
0
216
Mozilla/5.0
 (Linux; U; Android 4.2.2; ar-eg; NEO-X7-216A Build/JDQ39) AppleWebKit/534.30 
(KHTML, like Gecko) Version/4.0 Safari/534.30
none
\N
Online Breedband 
B.V.
eindhoven

NL
83.117.26.125

true
true
51.44
5.48
\N
5654
06
0
14182415864288907390719148304469
json
Live
122
\N
1418241586428
8907390719148304469
mobapp
0
0||
386
64.14.181.220

17597571-80a7-11e4-b1d6-52540004
1418169600
1418238000
img
top
\N
65772487
\N
0
0
politicsmobile_contentdatinggeneralringtones
66
na
Microsoft
exact_match
Windows
false
false
false
Generic
 Windows Mobile
320
240
Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; 
PPC)
none
\N
Aircel Ltd.
kolkata
mobile wireless
IN
101.218.108.20
mobile 
gateway
true
true
22.5697021484375
88.36968994140625
DISHNETIND
\N
28
0
14182415864284741441834639756033
xml
Live
76
\N
1418241586428
4741441834639756033
mobweb
0
0||
60
54.175.7.64
\N
\N
1418169600
1418238000

it`s exactly the reproducible in the HIVE-9080.

> OrcFileMergeOperator may create merge file that does not match properties of 
> input files
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-9067
>                 URL: https://issues.apache.org/jira/browse/HIVE-9067
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0, 0.15.0, 0.14.1
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Minor
>              Labels: Orc
>             Fix For: 1.0.0
>
>         Attachments: HIVE-9067.1.patch, HIVE-9067.2.patch
>
>
> OrcFileMergeOperator creates a new ORC file and appends the stripes from 
> smaller orc file. This new ORC file creation should retain the same 
> configuration as the small ORC files. Currently it does not set the orc row 
> index stride and file version. Also merging of stripe statistics to file 
> statistics was incorrect leading to issues like in HIVE-9080



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)