[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12907280#action_12907280
]
Doug Cutting commented on PIG-794:
--
Jeff, please instead use current trunk or the 1.4.0 build
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12906871#action_12906871
]
Doug Cutting commented on PIG-794:
--
Jeff, what version of Avro are you using?
Use Avro
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12907049#action_12907049
]
Jeff Zhang commented on PIG-794:
Doug, I am using avro trunk revision 988779
Use Avro
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12906671#action_12906671
]
Jeff Zhang commented on PIG-794:
Dmitriy,
In my patch I turn InternalMap as an avro array
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12905612#action_12905612
]
Dmitriy V. Ryaboy commented on PIG-794:
---
Doug and Scott will know better of course, but
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12905663#action_12905663
]
Doug Cutting commented on PIG-794:
--
Some quick comments on the new patch:
- you might
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12905775#action_12905775
]
Dmitriy V. Ryaboy commented on PIG-794:
---
Jeff, that's what I am saying -- since they are
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904551#action_12904551
]
Jeff Zhang commented on PIG-794:
I did some experiment on Avro, Avro_Storage_2.patch is the
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904615#action_12904615
]
Dmitriy V. Ryaboy commented on PIG-794:
---
Jeff, have you checkoed out Scott Carey's work
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904674#action_12904674
]
Scott Carey commented on PIG-794:
-
AVRO-592 creates an AvroStorage class for writing and
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904680#action_12904680
]
Scott Carey commented on PIG-794:
-
So a summary of the differences I can see quickly are:
h5.
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12904687#action_12904687
]
Doug Cutting commented on PIG-794:
--
A few comments about the attached code:
- is there a
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884577#action_12884577
]
Jeff Zhang commented on PIG-794:
We can leverage AvroInputFormat and AvroOutputFormat in Avro
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847594#action_12847594
]
Allen Wittenauer commented on PIG-794:
--
What is the latest on getting Avro support in
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847606#action_12847606
]
Alan Gates commented on PIG-794:
It depends on what you mean by support. As far as Pig using
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847607#action_12847607
]
Jeff Hammerbacher commented on PIG-794:
---
bq. Last time we tested the performance was
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847613#action_12847613
]
Alan Gates commented on PIG-794:
Jeff,
Beyond performance what do you see as the big wins of
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12847614#action_12847614
]
Dmitriy V. Ryaboy commented on PIG-794:
---
I'll take a crack at it.
Use Avro
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12729700#action_12729700
]
Alan Gates commented on PIG-794:
I agree with Doug's comments that it's better to use an API
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12723812#action_12723812
]
Alan Gates commented on PIG-794:
PIG-734 has been committed. This will allow this patch to
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12712397#action_12712397
]
Hong Tang commented on PIG-794:
---
- It appears that the code added a three-byte sync-mark \1\2\3
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12709608#action_12709608
]
Doug Cutting commented on PIG-794:
--
Looking at the patch, I have a few questions and remarks:
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12709078#action_12709078
]
Rakesh Setty commented on PIG-794:
--
Olga,
The reproduced the issue. It is because the schema
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12709081#action_12709081
]
Rakesh Setty commented on PIG-794:
--
I also noticed that the Avro codebase supports only
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12708701#action_12708701
]
Olga Natkovich commented on PIG-794:
I integrated the latest patch and run unit tests. All
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12707570#action_12707570
]
Rakesh Setty commented on PIG-794:
--
The new patch has unit tests. The comments are already in
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12707576#action_12707576
]
Rakesh Setty commented on PIG-794:
--
There was one important change I had to do in AvroStorage
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12707063#action_12707063
]
Olga Natkovich commented on PIG-794:
Rakesh,
Could you please convert the comments to
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706473#action_12706473
]
Rakesh Setty commented on PIG-794:
--
The attached jar files should go to the lib directory.
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706144#action_12706144
]
Rakesh Setty commented on PIG-794:
--
While trying to address the comment about eliminating the
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706197#action_12706197
]
Doug Cutting commented on PIG-794:
--
I think we have to ask the Avro team to support this
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706220#action_12706220
]
Rakesh Setty commented on PIG-794:
--
This works. Will update the patch.
Use Avro
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706226#action_12706226
]
Hadoop QA commented on PIG-794:
---
-1 overall. Here are the results of testing the latest
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706244#action_12706244
]
Olga Natkovich commented on PIG-794:
Doug, if there is no buffering then the position in
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706278#action_12706278
]
Olga Natkovich commented on PIG-794:
Hi Rakesh,
Thanks for the update. A few comments:
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12706284#action_12706284
]
Olga Natkovich commented on PIG-794:
One more thing: since we are adding avro library,
[
https://issues.apache.org/jira/browse/PIG-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12705351#action_12705351
]
Olga Natkovich commented on PIG-794:
Hi Rakesh,
Thanks for the patch. A few comments
37 matches
Mail list logo