I wouldn't recommend writing directly from Flume to Parquet. Parquet
can't guarantee that data is on disk until a file is closed, so you end
up with long-running transactions that back up into your file channel.
Plus, if you are writing to a partitioned dataset you end up with
several open
[
https://issues.apache.org/jira/browse/PARQUET-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Blue resolved PARQUET-369.
---
Resolution: Fixed
Assignee: Ryan Blue
Fix Version/s: format-2.3.1
> Shading SLF4J
[
https://issues.apache.org/jira/browse/PARQUET-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977266#comment-14977266
]
Stavros Kontopoulos commented on PARQUET-241:
-
Does this affect actual data order shown on
[
https://issues.apache.org/jira/browse/PARQUET-241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977329#comment-14977329
]
Ryan Blue commented on PARQUET-241:
---
[~skonto], I think that most formats are consistent by accident,
thanks to Ryan, i will do as you say.
lizhenm...@163.com
From: Ryan Blue
Date: 2015-10-28 00:07
To: dev
Subject: Re: how to convert text parquet in flume serialization
I wouldn't recommend writing directly from Flume to Parquet. Parquet
can't guarantee that data is on disk until a file is
[
https://issues.apache.org/jira/browse/PARQUET-140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977447#comment-14977447
]
Mark Grey commented on PARQUET-140:
---
I am wondering could address the converse case as well? (I have a
