Justin Coffey created HIVE-5783:
---
Summary: Native Parquet Support in Hive
Key: HIVE-5783
URL: https://issues.apache.org/jira/browse/HIVE-5783
Project: Hive
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey reassigned HIVE-5783:
---
Assignee: Justin Coffey
Native Parquet Support in Hive
--
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13820168#comment-13820168
]
Justin Coffey commented on HIVE-5783:
-
Thanks [~cwsteinbach] and [~ehans]. Regarding
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Fix Version/s: 0.11.0
Release Note: adds stored as parquet and setting parquet as the default
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: hive-0.11-parquet.patch
Native Parquet Support in Hive
--
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841666#comment-13841666
]
Justin Coffey commented on HIVE-5783:
-
[~appodictic], regarding the support being built
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13841681#comment-13841681
]
Justin Coffey commented on HIVE-5783:
-
{quote}
I think that was done before maven. I am
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843425#comment-13843425
]
Justin Coffey commented on HIVE-5783:
-
Hi [~cwsteinbach], so on the parquet-hive side,
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13843428#comment-13843428
]
Justin Coffey commented on HIVE-5783:
-
(sorry, errant trackpad submit on the last
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13844116#comment-13844116
]
Justin Coffey commented on HIVE-5783:
-
[~cwsteinbach] all sounds good. Regarding test
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13851661#comment-13851661
]
Justin Coffey commented on HIVE-5783:
-
Yes this is true. We are refactoring to merge
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924095#comment-13924095
]
Justin Coffey commented on HIVE-6414:
-
hello, I don't think these are related to the
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Attachment: HIVE-6414.3.patch
ParquetInputFormat provides data values that do not match the object
[
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950531#comment-13950531
]
Justin Coffey commented on HIVE-6757:
-
Owen, the solution your proposing means that
[
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13950996#comment-13950996
]
Justin Coffey commented on HIVE-6757:
-
I guess my point is simply that early adopters
[
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13955121#comment-13955121
]
Justin Coffey commented on HIVE-6757:
-
I can +1 [~brocknoland]'s solution if that flies
[
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959178#comment-13959178
]
Justin Coffey commented on HIVE-6757:
-
I find that to be an acceptable compromise.
[
https://issues.apache.org/jira/browse/HIVE-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13961892#comment-13961892
]
Justin Coffey commented on HIVE-6757:
-
much appreciated Harish!
Remove deprecated
[
https://issues.apache.org/jira/browse/HIVE-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13965279#comment-13965279
]
Justin Coffey commented on HIVE-6784:
-
-1 on this patch.
Looping on the arraywriteable
[
https://issues.apache.org/jira/browse/HIVE-6784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13966341#comment-13966341
]
Justin Coffey commented on HIVE-6784:
-
You've cited a lazy serde. Parquet is not lazy.
Justin Coffey created HIVE-6920:
---
Summary: Parquet Serde Simplification
Key: HIVE-6920
URL: https://issues.apache.org/jira/browse/HIVE-6920
Project: Hive
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6920:
Attachment: HIVE-6920.patch
Parquet Serde Simplification
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6920:
Release Note:
- Removed unused serde stats
- Simplified initialize code
- Renamed test class to
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6920:
Release Note:
- Removed unused serde stats
- Simplified initialize code
- Renamed test class to
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13972790#comment-13972790
]
Justin Coffey commented on HIVE-6920:
-
cc: [~brocknoland] [~xuefuz]
Parquet Serde
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981072#comment-13981072
]
Justin Coffey commented on HIVE-6920:
-
It's actually mostly just code reduction. Here's
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984183#comment-13984183
]
Justin Coffey commented on HIVE-6920:
-
bump?
I'd like to build off of this for a bug
Justin Coffey created HIVE-6994:
---
Summary: parquet-hive createArray strips null elements
Key: HIVE-6994
URL: https://issues.apache.org/jira/browse/HIVE-6994
Project: Hive
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Status: Patch Available (was: Open)
This patch fixes the issue in ParquetHiveSerDe, but there may
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Attachment: HIVE-6994.patch
parquet-hive createArray strips null elements
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Description:
The createArray method in ParquetHiveSerDe strips null values from resultant
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6920:
Status: Open (was: Patch Available)
Please see superceding issue here: #HIVE-6994
Parquet Serde
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985772#comment-13985772
]
Justin Coffey commented on HIVE-6994:
-
review board link:
[
https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13985777#comment-13985777
]
Justin Coffey commented on HIVE-6920:
-
btw, in the superceding patch, I killed the pom
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13989688#comment-13989688
]
Justin Coffey commented on HIVE-6994:
-
Hello, is it just me or does it look like the
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Attachment: HIVE-6994-1.patch
updated patch after rebasing against the trunk. it applies for me :)
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Attachment: HIVE-6994.2.patch
Updated based on comments on review board and fixed to include the
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6994:
Attachment: HIVE-6994.3.patch
The failed tests are unrelated to the patch--submitting a rebased
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: parquet-hive.patch
Native Parquet Support in Hive
--
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874626#comment-13874626
]
Justin Coffey commented on HIVE-5783:
-
After much delay, here is the patch. This
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874666#comment-13874666
]
Justin Coffey commented on HIVE-5783:
-
[~rusanu]: like so?
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875930#comment-13875930
]
Justin Coffey commented on HIVE-5783:
-
Hi [~cwsteinbach]. Actually, that looks like
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: (was: parquet-hive.patch)
Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: (was: hive-0.11-parquet.patch)
Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: HIVE-5783.patch
without license or author tags.
Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: (was: HIVE-5783.patch)
Native Parquet Support in Hive
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: HIVE-5783.patch
this is the good one. had a final dependency to clean up.
Native
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13876712#comment-13876712
]
Justin Coffey commented on HIVE-5783:
-
Sorry for the spam in posts. Latest patch is
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13877509#comment-13877509
]
Justin Coffey commented on HIVE-5783:
-
[~leftylev], if you'd like I can give this a
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879960#comment-13879960
]
Justin Coffey commented on HIVE-5783:
-
We have unfortunately found a bug in
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-5783:
Attachment: HIVE-5783.patch
The updated patch. This fixes incorrect behavior when using
[
https://issues.apache.org/jira/browse/HIVE-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896303#comment-13896303
]
Justin Coffey commented on HIVE-5783:
-
Thanks to all, and especially [~brocknoland] for
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13899284#comment-13899284
]
Justin Coffey commented on HIVE-6414:
-
I'll investigate.
ParquetInputFormat provides
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey reassigned HIVE-6414:
---
Assignee: Justin Coffey
ParquetInputFormat provides data values that do not match the object
[
https://issues.apache.org/jira/browse/HIVE-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904423#comment-13904423
]
Justin Coffey commented on HIVE-6456:
-
good to go. thanks for the fast work!
Improve
[
https://issues.apache.org/jira/browse/HIVE-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905450#comment-13905450
]
Justin Coffey commented on HIVE-6456:
-
brock and I had the same thought offline. Not
Justin Coffey created HIVE-6463:
---
Summary: unit test for evoloving schema in parquet files
Key: HIVE-6463
URL: https://issues.apache.org/jira/browse/HIVE-6463
Project: Hive
Issue Type: Test
[
https://issues.apache.org/jira/browse/HIVE-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905459#comment-13905459
]
Justin Coffey commented on HIVE-6456:
-
done and linked.
Implement Parquet schema
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Fix Version/s: 0.13.0
Affects Version/s: 0.13.0
Status: Patch Available
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Attachment: HIVE-6414.patch
Credit should be given to Remy Pecqueur r.pecqu...@criteo.com
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911585#comment-13911585
]
Justin Coffey commented on HIVE-6414:
-
Hi Szehon, I worked off of the trunk on this.
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13911587#comment-13911587
]
Justin Coffey commented on HIVE-6414:
-
Oh, and we don't appear to need the order by for
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Attachment: HIVE-6414.2.patch
Updated patch with working unit and qtests applicable to trunk
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13912643#comment-13912643
]
Justin Coffey commented on HIVE-6414:
-
[~xuefuz] ok will recheck qtest and resubmit
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Attachment: HIVE-6414.3.patch
Update patch based on comments from Xuefu.
ParquetInputFormat
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002985#comment-14002985
]
Justin Coffey commented on HIVE-6994:
-
Thanks Shzehon!
parquet-hive createArray
[
https://issues.apache.org/jira/browse/HIVE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004924#comment-14004924
]
Justin Coffey commented on HIVE-6994:
-
hmmm... good catch. It didn't get picked up by
67 matches
Mail list logo