ASF GitHub Bot commented on ARROW-2090:

Posnet opened a new pull request #1559: ARROW-2090: Add context methods to 
URL: https://github.com/apache/arrow/pull/1559
   Added the `__enter__` and `__exit__` methods to the ParquetWriter
   Class. This allows you to write code in the following style:
   with ParquetWriter(foo, schema) as writer:
   And the ParquetWriter context object will handle calling the close
   method for you when the with block exits.  It propagates errors
   in line with previous behavior. I have also updated the module
   level `write_table` method to use the new `with` style.
   And it maintains it's existing behavior of trying to remove
   a partial file if an exception is encountered.
   The reason for this change, was a bug I encountered while
   using the `ParquetWriter` object, where I wasn't closing
   the writer correctly in the event of an exception, which resulted
   in partially written parquet files. Adding the ability to use
   `with` syntax will reduce the chances of a similar misuse
   while still maintaing backwards compatibility.

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Add context manager methods to ParquetWriter
> --------------------------------------------
>                 Key: ARROW-2090
>                 URL: https://issues.apache.org/jira/browse/ARROW-2090
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Python
>            Reporter: Alec Posney
>            Priority: Minor
>              Labels: pull-request-available
> Add the ability to use python {{with}} syntax on the {{ParquetWriter}} object.
> For example:
> {code:python}
> with pq.ParquetWriter(foo, schema) as writer:
>     writer.write_table(table)
> {code}
> The benefit of this syntax is that it removes the chances of you writing out 
> a partial (invalid) parquet file, which is currently possible if you forget 
> to call the close method, or more likely the close method is not called due 
> to an exception being thrown. 
> It should still be possible to use the previous syntax for backwards 
> compatibility and fine grained control reasons.
> Similarly, the {{parquet}} module level {{write_table}} method should be able 
> to use the new syntax without changing previous behaviour. 

This message was sent by Atlassian JIRA

Reply via email to