Author: cutting
Date: Thu Nov 6 23:18:10 2014
New Revision: 1637264
URL: http://svn.apache.org/r1637264
Log:
AVRO-1302. Python: Update documentation to open files as binary to prevent EOL
substitution. Contributed by Lars Francke.
Modified:
avro/trunk/CHANGES.txt
avro/trunk/doc/src/content/xdocs/gettingstartedpython.xml
Modified: avro/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/avro/trunk/CHANGES.txt?rev=1637264&r1=1637263&r2=1637264&view=diff
==============================================================================
--- avro/trunk/CHANGES.txt (original)
+++ avro/trunk/CHANGES.txt Thu Nov 6 23:18:10 2014
@@ -55,6 +55,9 @@ Trunk (not yet released)
AVRO-1489. Java: Avro fails to build with OpenJDK 8. (Ricardo Arguello via
tomwhite)
+ AVRO-1302. Python: Update documentation to open files as binary to
+ prevent EOL substitution. (Lars Francke via cutting)
+
Avro 1.7.7 (23 July 2014)
NEW FEATURES
Modified: avro/trunk/doc/src/content/xdocs/gettingstartedpython.xml
URL:
http://svn.apache.org/viewvc/avro/trunk/doc/src/content/xdocs/gettingstartedpython.xml?rev=1637264&r1=1637263&r2=1637264&view=diff
==============================================================================
--- avro/trunk/doc/src/content/xdocs/gettingstartedpython.xml (original)
+++ avro/trunk/doc/src/content/xdocs/gettingstartedpython.xml Thu Nov 6
23:18:10 2014
@@ -136,14 +136,14 @@ import avro.schema
from avro.datafile import DataFileReader, DataFileWriter
from avro.io import DatumReader, DatumWriter
-schema = avro.schema.parse(open("user.avsc").read())
+schema = avro.schema.parse(open("user.avsc", "rb").read())
-writer = DataFileWriter(open("users.avro", "w"), DatumWriter(), schema)
+writer = DataFileWriter(open("users.avro", "wb"), DatumWriter(), schema)
writer.append({"name": "Alyssa", "favorite_number": 256})
writer.append({"name": "Ben", "favorite_number": 7, "favorite_color": "red"})
writer.close()
-reader = DataFileReader(open("users.avro", "r"), DatumReader())
+reader = DataFileReader(open("users.avro", "rb"), DatumReader())
for user in reader:
print user
reader.close()
@@ -154,10 +154,18 @@ reader.close()
{u'favorite_color': u'red', u'favorite_number': 7, u'name': u'Ben'}
</source>
<p>
+ Do make sure that you open your files in binary mode (i.e. using the
modes
+ <code>wb</code> or <code>rb</code> respectively). Otherwise you might
+ generate corrupt files due to
+ <a href="http://docs.python.org/library/functions.html#open";>
+ automatic replacement</a> of newline characters with the
+ platform-specific representations.
+ </p>
+ <p>
Let's take a closer look at what's going on here.
</p>
<source>
-schema = avro.schema.parse(open("user.avsc").read())
+schema = avro.schema.parse(open("user.avsc", "rb").read())
</source>
<p>
<code>avro.schema.parse</code> takes a string containing a JSON schema
@@ -167,7 +175,7 @@ schema = avro.schema.parse(open("user.av
user.avsc schema file here.
</p>
<source>
-writer = DataFileWriter(open("users.avro", "w"), DatumWriter(), schema)
+writer = DataFileWriter(open("users.avro", "wb"), DatumWriter(), schema)
</source>
<p>
We create a <code>DataFileWriter</code>, which we'll use to write
@@ -201,7 +209,7 @@ writer.append({"name": "Ben", "favorite_
ignored.
</p>
<source>
-reader = DataFileReader(open("users.avro", "r"), DatumReader())
+reader = DataFileReader(open("users.avro", "rb"), DatumReader())
</source>
<p>
We open the file again, this time for reading back from disk. We use
