This is an automated email from the git hooks/post-receive script. sebastic pushed a commit to branch master in repository pyshp.
commit 43009b207a964e9019f47a4fd59695eb3c8a531e Author: Bas Couwenberg <[email protected]> Date: Sun Apr 30 11:32:39 2017 +0200 Imported Upstream version 1.2.11+ds --- .travis.yml | 14 - PKG-INFO | 761 ++++++++++++++++++++++++++++++++++++++++++++ README.md | 330 +++++++++++++------ changelog.txt | 12 + setup.cfg | 5 + setup.py | 2 +- shapefile.py | 217 ++++++++----- shapefiles/test/dtype.dbf | Bin 0 -> 77 bytes shapefiles/test/dtype.shp | Bin 0 -> 160 bytes shapefiles/test/dtype.shx | Bin 0 -> 140 bytes shapefiles/test/line.dbf | Bin 0 -> 2122 bytes shapefiles/test/line.shp | Bin 0 -> 3452 bytes shapefiles/test/line.shx | Bin 0 -> 300 bytes shapefiles/test/point.dbf | Bin 0 -> 299 bytes shapefiles/test/point.shp | Bin 0 -> 156 bytes shapefiles/test/point.shx | Bin 0 -> 116 bytes shapefiles/test/polygon.dbf | Bin 0 -> 198 bytes shapefiles/test/polygon.shp | Bin 0 -> 324 bytes shapefiles/test/polygon.shx | Bin 0 -> 116 bytes 19 files changed, 1148 insertions(+), 193 deletions(-) diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index ab4803a..0000000 --- a/.travis.yml +++ /dev/null @@ -1,14 +0,0 @@ -language: python -python: -- '2.7' -script: -- touch foo -deploy: - provider: pypi - user: jlawhead - password: - secure: w1ErvS4wRNUVlWOtisWIC9BZiV6qecAKUN8/KUHxpEPRe54DKYgd9VMPlEktHIEFqHiUw0QH3ObJZCghDDjbu9CliGU1plIOlBbxaX2EL6sAlmmZuwm2Xw2NplVclKI2QHyFLlABlKkH5eoFVFoMxdXMTGHAMcubk+EJoKEVHsGDhhHGaT1T8I8gV1kIzpYBe2/3b5Nol1/lKpFqHet017IJ8TZWziHsx91zqqDj9+T0cX3QllkZNn4PtY4KojcszTQyRQGwcEV9mzL/npmHtIqEIhQCqRc5vv+/wMU7aFfpA8NZEIcxb79nwiuSfmA6/iONBzHFJYPIws7cWJpBfTHYHX8uxUq9Nh4FTIPutIO92cx8xp5+ftTZRXxi4Wi6ESycMp/erQHgGPlvjkgzwWFwNCkMCCGAGZfH2aaSOBSAxf/s8DBGWQ9OzqtWxirHy/hzjdleWduFii1aw0I5nxjgvKzgM/je [...] - on: - tags: true - distributions: sdist bdist_wheel - repo: GeospatialPython/pyshp diff --git a/PKG-INFO b/PKG-INFO new file mode 100644 index 0000000..66dc8fc --- /dev/null +++ b/PKG-INFO @@ -0,0 +1,761 @@ +Metadata-Version: 1.1 +Name: pyshp +Version: 1.2.11 +Summary: Pure Python read/write support for ESRI Shapefile format +Home-page: https://github.com/GeospatialPython/pyshp +Author: Joel Lawhead +Author-email: [email protected] +License: MIT +Download-URL: https://github.com/GeospatialPython/pyshp/archive/1.2.10.tar.gz +Description: # PyShp + + The Python Shapefile Library (pyshp) reads and writes ESRI Shapefiles in pure Python. + +  + + [](https://travis-ci.org/GeospatialPython/pyshp) + + ## Contents + + [Overview](#overview) + + [Examples](#examples) + - [Reading Shapefiles](#reading-shapefiles) + - [Reading Shapefiles from File-Like Objects](#reading-shapefiles-from-file-like-objects) + - [Reading Geometry](#reading-geometry) + - [Reading Records](#reading-records) + - [Reading Geometry and Records Simultaneously](#reading-geometry-and-records-simultaneously) + - [Writing Shapefiles](#writing-shapefiles) + - [Setting the Shape Type](#setting-the-shape-type) + - [Geometry and Record Balancing](#geometry-and-record-balancing) + - [Adding Geometry](#adding-geometry) + - [Creating Attributes](#creating-attributes) + - [File Names](#file-names) + - [Saving to File-Like Objects](#saving-to-file-like-objects) + - [Editing Shapefiles](#editing-shapefiles) + - [Geometry and Record Balancing](#geometry-and-record-balancing) + - [Python \_\_geo_interface\_\_](#python-\_\_geo\_interface\_\_) + - [Testing](#testing) + + # Overview + + The Python Shapefile Library (pyshp) provides read and write support for the + Esri Shapefile format. The Shapefile format is a popular Geographic + Information System vector data format created by Esri. For more information + about this format please read the well-written "ESRI Shapefile Technical + Description - July 1998" located at [http://www.esri.com/library/whitepapers/p + dfs/shapefile.pdf](http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf) + . The Esri document describes the shp and shx file formats. However a third + file format called dbf is also required. This format is documented on the web + as the "XBase File Format Description" and is a simple file-based database + format created in the 1960's. For more on this specification see: [http://www.clicketyclick.dk/databases/xbase/format/index.html](http://www.clicketyclick.d + k/databases/xbase/format/index.html) + + Both the Esri and XBase file-formats are very simple in design and memory + efficient which is part of the reason the shapefile format remains popular + despite the numerous ways to store and exchange GIS data available today. + + Pyshp is compatible with Python 2.4-3.x. + + This document provides examples for using pyshp to read and write shapefiles. However + many more examples are continually added to the pyshp wiki on GitHub, the blog [http://GeospatialPython.com](http://GeospatialPython.com), + and by searching for pyshp on [https://gis.stackexchange.com](https://gis.stackexchange.com). + + Currently the sample census blockgroup shapefile referenced in the examples is available on the GitHub project site at + [https://github.com/GeospatialPython/pyshp](https://github.com/GeospatialPython/pyshp). These + examples are straight-forward and you can also easily run them against your + own shapefiles with minimal modification. + + Important: If you are new to GIS you should read about map projections. + Please visit: [https://github.com/GeospatialPython/pyshp/wiki/Map-Projections](https://github.com/GeospatialPython/pyshp/wiki/Map-Projections) + + I sincerely hope this library eliminates the mundane distraction of simply + reading and writing data, and allows you to focus on the challenging and FUN + part of your geospatial project. + + # Examples + + Before doing anything you must import the library. + + >>> import shapefile + + The examples below will use a shapefile created from the U.S. Census Bureau + Blockgroups data set near San Francisco, CA and available in the git + repository of the pyshp GitHub site. + + ## Reading Shapefiles + + To read a shapefile create a new "Reader" object and pass it the name of an + existing shapefile. The shapefile format is actually a collection of three + files. You specify the base filename of the shapefile or the complete filename + of any of the shapefile component files. + + + >>> sf = shapefile.Reader("shapefiles/blockgroups") + + OR + + + >>> sf = shapefile.Reader("shapefiles/blockgroups.shp") + + OR + + + >>> sf = shapefile.Reader("shapefiles/blockgroups.dbf") + + OR any of the other 5+ formats which are potentially part of a shapefile. The + library does not care about file extensions. + + ### Reading Shapefiles from File-Like Objects + + You can also load shapefiles from any Python file-like object using keyword + arguments to specify any of the three files. This feature is very powerful and + allows you to load shapefiles from a url, from a zip file, serialized object, + or in some cases a database. + + + >>> myshp = open("shapefiles/blockgroups.shp", "rb") + >>> mydbf = open("shapefiles/blockgroups.dbf", "rb") + >>> r = shapefile.Reader(shp=myshp, dbf=mydbf) + + Notice in the examples above the shx file is never used. The shx file is a + very simple fixed-record index for the variable length records in the shp + file. This file is optional for reading. If it's available pyshp will use the + shx file to access shape records a little faster but will do just fine without + it. + + ### Reading Geometry + + A shapefile's geometry is the collection of points or shapes made from + vertices and implied arcs representing physical locations. All types of + shapefiles just store points. The metadata about the points determine how they + are handled by software. + + You can get the a list of the shapefile's geometry by calling the shapes() + method. + + + >>> shapes = sf.shapes() + + The shapes method returns a list of Shape objects describing the geometry of + each shape record. + + + >>> len(shapes) + 663 + + You can iterate through the shapefile's geometry using the iterShapes() + method. + + + >>> len(list(sf.iterShapes())) + 663 + + Each shape record contains the following attributes: + + + >>> for name in dir(shapes[3]): + ... if not name.startswith('__'): + ... name + 'bbox' + 'parts' + 'points' + 'shapeType' + + * shapeType: an integer representing the type of shape as defined by the + shapefile specification. + + + >>> shapes[3].shapeType + 5 + + * bbox: If the shape type contains multiple points this tuple describes the + lower left (x,y) coordinate and upper right corner coordinate creating a + complete box around the points. If the shapeType is a + Null (shapeType == 0) then an AttributeError is raised. + + + >>> # Get the bounding box of the 4th shape. + >>> # Round coordinates to 3 decimal places + >>> bbox = shapes[3].bbox + >>> ['%.3f' % coord for coord in bbox] + ['-122.486', '37.787', '-122.446', '37.811'] + + * parts: Parts simply group collections of points into shapes. If the shape + record has multiple parts this attribute contains the index of the first + point of each part. If there is only one part then a list containing 0 is + returned. + + >>> shapes[3].parts + [0] + + * points: The points attribute contains a list of tuples containing an + (x,y) coordinate for each point in the shape. + + >>> len(shapes[3].points) + 173 + >>> # Get the 8th point of the fourth shape + >>> # Truncate coordinates to 3 decimal places + >>> shape = shapes[3].points[7] + >>> ['%.3f' % coord for coord in shape] + ['-122.471', '37.787'] + + To read a single shape by calling its index use the shape() method. The index + is the shape's count from 0. So to read the 8th shape record you would use its + index which is 7. + + + >>> s = sf.shape(7) + + >>> # Read the bbox of the 8th shape to verify + >>> # Round coordinates to 3 decimal places + >>> ['%.3f' % coord for coord in s.bbox] + ['-122.450', '37.801', '-122.442', '37.808'] + + ### Reading Records + + A record in a shapefile contains the attributes for each shape in the + collection of geometry. Records are stored in the dbf file. The link between + geometry and attributes is the foundation of all geographic information systems. + This critical link is implied by the order of shapes and corresponding records + in the shp geometry file and the dbf attribute file. + + The field names of a shapefile are available as soon as you read a shapefile. + You can call the "fields" attribute of the shapefile as a Python list. Each + field is a Python list with the following information: + + * Field name: the name describing the data at this column index. + * Field type: the type of data at this column index. Types can be: Character, + Numbers, Longs, Dates, or Memo. The "Memo" type has no meaning within a + GIS and is part of the xbase spec instead. + * Field length: the length of the data found at this column index. Older GIS + software may truncate this length to 8 or 11 characters for "Character" + fields. + * Decimal length: the number of decimal places found in "Number" fields. + + To see the fields for the Reader object above (sf) call the "fields" + attribute: + + + >>> fields = sf.fields + + >>> assert fields == [("DeletionFlag", "C", 1, 0), ["AREA", "N", 18, 5], + ... ["BKG_KEY", "C", 12, 0], ["POP1990", "N", 9, 0], ["POP90_SQMI", "N", 10, 1], + ... ["HOUSEHOLDS", "N", 9, 0], + ... ["MALES", "N", 9, 0], ["FEMALES", "N", 9, 0], ["WHITE", "N", 9, 0], + ... ["BLACK", "N", 8, 0], ["AMERI_ES", "N", 7, 0], ["ASIAN_PI", "N", 8, 0], + ... ["OTHER", "N", 8, 0], ["HISPANIC", "N", 8, 0], ["AGE_UNDER5", "N", 8, 0], + ... ["AGE_5_17", "N", 8, 0], ["AGE_18_29", "N", 8, 0], ["AGE_30_49", "N", 8, 0], + ... ["AGE_50_64", "N", 8, 0], ["AGE_65_UP", "N", 8, 0], + ... ["NEVERMARRY", "N", 8, 0], ["MARRIED", "N", 9, 0], ["SEPARATED", "N", 7, 0], + ... ["WIDOWED", "N", 8, 0], ["DIVORCED", "N", 8, 0], ["HSEHLD_1_M", "N", 8, 0], + ... ["HSEHLD_1_F", "N", 8, 0], ["MARHH_CHD", "N", 8, 0], + ... ["MARHH_NO_C", "N", 8, 0], ["MHH_CHILD", "N", 7, 0], + ... ["FHH_CHILD", "N", 7, 0], ["HSE_UNITS", "N", 9, 0], ["VACANT", "N", 7, 0], + ... ["OWNER_OCC", "N", 8, 0], ["RENTER_OCC", "N", 8, 0], + ... ["MEDIAN_VAL", "N", 7, 0], ["MEDIANRENT", "N", 4, 0], + ... ["UNITS_1DET", "N", 8, 0], ["UNITS_1ATT", "N", 7, 0], ["UNITS2", "N", 7, 0], + ... ["UNITS3_9", "N", 8, 0], ["UNITS10_49", "N", 8, 0], + ... ["UNITS50_UP", "N", 8, 0], ["MOBILEHOME", "N", 7, 0]] + + You can get a list of the shapefile's records by calling the records() method: + + + >>> records = sf.records() + + >>> len(records) + 663 + + Similar to the geometry methods, you can iterate through dbf records using the + iterRecords() method. + + + >>> len(list(sf.iterRecords())) + 663 + + Each record is a list containing an attribute corresponding to each field in + the field list. + + For example in the 4th record of the blockgroups shapefile the 2nd and 3rd + fields are the blockgroup id and the 1990 population count of that San + Francisco blockgroup: + + + >>> records[3][1:3] + ['060750601001', 4715] + + To read a single record call the record() method with the record's index: + + + >>> rec = sf.record(3) + + >>> rec[1:3] + ['060750601001', 4715] + + ### Reading Geometry and Records Simultaneously + + You may want to examine both the geometry and the attributes for a record at + the same time. The shapeRecord() and shapeRecords() method let you do just + that. + + Calling the shapeRecords() method will return the geometry and attributes for + all shapes as a list of ShapeRecord objects. Each ShapeRecord instance has a + "shape" and "record" attribute. The shape attribute is a ShapeRecord object as + discussed in the first section "Reading Geometry". The record attribute is a + list of field values as demonstrated in the "Reading Records" section. + + + >>> shapeRecs = sf.shapeRecords() + + Let's read the blockgroup key and the population for the 4th blockgroup: + + + >>> shapeRecs[3].record[1:3] + ['060750601001', 4715] + + Now let's read the first two points for that same record: + + + >>> points = shapeRecs[3].shape.points[0:2] + + >>> len(points) + 2 + + The shapeRecord() method reads a single shape/record pair at the specified index. + To get the 4th shape record from the blockgroups shapefile use the third index: + + + >>> shapeRec = sf.shapeRecord(3) + + The blockgroup key and population count: + + + >>> shapeRec.record[1:3] + ['060750601001', 4715] + + >>> points = shapeRec.shape.points[0:2] + + >>> len(points) + 2 + + There is also an iterShapeRecords() method to iterate through large files: + + + >>> shapeRecs = sf.iterShapeRecords() + >>> for shapeRec in shapeRecs: + ... # do something here + ... pass + + + ## Writing Shapefiles + + PyShp tries to be as flexible as possible when writing shapefiles while + maintaining some degree of automatic validation to make sure you don't + accidentally write an invalid file. + + PyShp can write just one of the component files such as the shp or dbf file + without writing the others. So in addition to being a complete shapefile + library, it can also be used as a basic dbf (xbase) library. Dbf files are a + common database format which are often useful as a standalone simple database + format. And even shp files occasionally have uses as a standalone format. Some + web-based GIS systems use an user-uploaded shp file to specify an area of + interest. Many precision agriculture chemical field sprayers also use the shp + format as a control file for the sprayer system (usually in combination with + custom database file formats). + + To create a shapefile you add geometry and/or attributes using methods in the + Writer class until you are ready to save the file. + + Create an instance of the Writer class to begin creating a shapefile: + + + >>> w = shapefile.Writer() + + ### Setting the Shape Type + + The shape type defines the type of geometry contained in the shapefile. All of + the shapes must match the shape type setting. + + Shape types are represented by numbers between 0 and 31 as defined by the + shapefile specification. It is important to note that numbering system has + several reserved numbers which have not been used yet therefore the numbers of + the existing shape types are not sequential. + + There are three ways to set the shape type: + * Set it when creating the class instance. + * Set it by assigning a value to an existing class instance. + * Set it automatically to the type of the first non-null shape by saving the shapefile. + + To manually set the shape type for a Writer object when creating the Writer: + + + >>> w = shapefile.Writer(shapeType=1) + + >>> w.shapeType + 1 + + OR you can set it after the Writer is created: + + + >>> w.shapeType = 3 + + >>> w.shapeType + 3 + + ### Geometry and Record Balancing + + Because every shape must have a corresponding record it is critical that the + number of records equals the number of shapes to create a valid shapefile. You + must take care to add records and shapes in the same order so that the record + data lines up with the geometry data. For example: + + + >>> w.field("field1", "C") + >>> w.field("field2", "C") + >>> w.record("row", "one") + >>> w.record("row", "two") + >>> w.point(1, 1) + >>> w.point(2, 2) + + ### Adding Geometry + + Geometry is added using one of three methods: "null", "point", or "poly". The + "null" method is used for null shapes, "point" is used for point shapes, "line" for lines, and + "poly" is used for polygons and everything else. + + **Adding a Point shape** + + Point shapes are added using the "point" method. A point is specified by an x, + y, and optional z (elevation) and m (measure) value. + + + >>> w = shapefile.Writer(shapefile.POINTM) + + >>> w.point(122, 37) # No elevation or measure values + + >>> w.shapes()[0].points + [[122, 37, 0, 0]] + + >>> w.point(118, 36, 4, 8) + + >>> w.shapes()[1].points + [[118, 36, 4, 8]] + + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/point') + + **Adding a Polygon shape** + + Shapefile polygons must have at + least 4 points and the last point must be the same as the first. PyShp + automatically enforces closed polygons. + + + >>> w = shapefile.Writer() + + >>> w.poly(parts=[[[122,37,4,9], [117,36,3,4]], [[115,32,8,8], + ... [118,20,6,4], [113,24]]]) + + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/polygon') + + **Adding a Line shape** + + A line must have at least two points. + Because of the similarities between polygon and line types it is possible to create + a line shape using either the "line" or "poly" method. + + + >>> w = shapefile.Writer() + + >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]]) + >>> w.poly(parts=[[[1,3],[5,3]]], shapeType=shapefile.POLYLINE) + + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/polygon') + + **Adding a Null shape** + + Because Null shape types (shape type 0) have no geometry the "null" method is + called without any arguments. This type of shapefile is rarely used but it is valid. + + + >>> w = shapefile.Writer() + + >>> w.null() + + The writer object's shapes list will now have one null shape: + + + >>> assert w.shapes()[0].shapeType == shapefile.NULL + + ### Creating Attributes + + Creating attributes involves two steps. Step 1 is to create fields to contain + attribute values and step 2 is to populate the fields with values for each + shape record. + + There are several different field types, all of which support storing None values as NULL. + + Text fields are created using the 'C' type, and the third 'size' argument can be customized to the expected + length of text values to save space: + + + >>> w = shapefile.Writer() + >>> w.field('TEXT', 'C') + >>> w.field('SHORT_TEXT', 'C', size=5) + >>> w.field('LONG_TEXT', 'C', size=250) + >>> w.null() + >>> w.record('Hello', 'World', 'World'*50) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == ['Hello', 'World', 'World'*50] + + Date fields are created using the 'D' type, and can be created using either + date objects, lists, or a YYYYMMDD formatted string. + Field length or decimal have no impact on this type: + + + >>> from datetime import date + >>> w = shapefile.Writer() + >>> w.field('DATE', 'D') + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.record(date(1998,1,30)) + >>> w.record([1998,1,30]) + >>> w.record('19980130') + >>> w.record(None) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == [date(1998,1,30)] + >>> assert r.record(1) == [date(1998,1,30)] + >>> assert r.record(2) == [date(1998,1,30)] + >>> assert r.record(3) == [None] + + Numeric fields are created using the 'N' type (or the 'F' type, which is exactly the same). + By default the fourth decimal argument is set to zero, essentially creating an integer field. + To store floats you must set the decimal argument to the precision of your choice. + To store very large numbers you must increase the field length size to the total number of digits + (including comma and minus). + + + >>> w = shapefile.Writer() + >>> w.field('INT', 'N') + >>> w.field('LOWPREC', 'N', decimal=2) + >>> w.field('MEDPREC', 'N', decimal=10) + >>> w.field('HIGHPREC', 'N', decimal=30) + >>> w.field('FTYPE', 'F', decimal=10) + >>> w.field('LARGENR', 'N', 101) + >>> nr = 1.3217328 + >>> w.null() + >>> w.null() + >>> w.record(INT=int(nr), LOWPREC=nr, MEDPREC=nr, HIGHPREC=-3.2302e-25, FTYPE=nr, LARGENR=int(nr)*10**100) + >>> w.record(None, None, None, None, None, None) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == [1, 1.32, 1.3217328, -3.2302e-25, 1.3217328, 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] + >>> assert r.record(1) == [None, None, None, None, None, None] + + + Finally, we can create boolean fields by setting the type to 'L'. + This field can take True or False values, or 1 (True) or 0 (False). + None is interpreted as missing. + + + >>> w = shapefile.Writer() + >>> w.field('BOOLEAN', 'L') + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.record(True) + >>> w.record(1) + >>> w.record(False) + >>> w.record(0) + >>> w.record(None) + >>> w.record("Nonesense") + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> r.record(0) + [True] + >>> r.record(1) + [True] + >>> r.record(2) + [False] + >>> r.record(3) + [False] + >>> r.record(4) + [None] + >>> r.record(5) + [None] + + You can also add attributes using keyword arguments where the keys are field names. + + + >>> w = shapefile.Writer() + >>> w.field('FIRST_FLD','C','40') + >>> w.field('SECOND_FLD','C','40') + >>> w.record('First', 'Line') + >>> w.record(FIRST_FLD='First', SECOND_FLD='Line') + >>> assert w.records[0] == w.records[1] + + + + ### File Names + + File extensions are optional when reading or writing shapefiles. If you specify + them PyShp ignores them anyway. When you save files you can specify a base + file name that is used for all three file types. Or you can specify a name for + one or more file types. In that case, any file types not assigned will not + save and only file types with file names will be saved. If you do not specify + any file names (i.e. save()), then a unique file name is generated with the + prefix "shapefile_" followed by random characters which is used for all three + files. The unique file name is returned as a string. + + + >>> targetName = w.save() + >>> assert("shapefile_" in targetName) + + ### Saving to File-Like Objects + + Just as you can read shapefiles from python file-like objects you can also + write them. + + + >>> try: + ... from StringIO import StringIO + ... except ImportError: + ... from io import BytesIO as StringIO + >>> shp = StringIO() + >>> shx = StringIO() + >>> dbf = StringIO() + >>> w.saveShp(shp) + >>> w.saveShx(shx) + >>> w.saveDbf(dbf) + >>> # Normally you would call the "StringIO.getvalue()" method on these objects. + >>> shp = shx = dbf = None + + ## Editing Shapefiles + + The Editor class attempts to make changing existing shapefiles easier by + handling the reading and writing details behind the scenes. This class is + experimental, has lots of issues, and should be avoided for production use. *You can do the same + thing by reading a shapefile into memory, making changes to the python objects, + and write out a new shapefile with the same or different name.* + + Let's add shapes to existing shapefiles: + + Add a point to a point shapefile: + + + >>> e = shapefile.Editor(shapefile="shapefiles/test/point.shp") + >>> e.point(0,0,10,2) + >>> e.record("Appended","Point") + >>> e.save('shapefiles/test/point') + + Add a new line to a line shapefile: + + + >>> e = shapefile.Editor(shapefile="shapefiles/test/line.shp") + >>> e.line(parts=[[[10,5],[15,5],[15,1],[13,3],[11,1]]]) + >>> e.record('Appended','Line') + >>> e.save('shapefiles/test/line') + + Add a new polygon to a polygon shapefile: + + + >>> e = shapefile.Editor(shapefile="shapefiles/test/polygon.shp") + >>> e.poly(parts=[[[5.1,5],[9.9,5],[9.9,1],[7.5,3],[5.1,1]]]) + >>> e.record("Appended","Polygon") + >>> e.save('shapefiles/test/polygon') + + Remove the first point in each shapefile - for a point shapefile that is the + first shape and record": + + + >>> e = shapefile.Editor(shapefile="shapefiles/test/point.shp") + >>> e.delete(0) + >>> e.save('shapefiles/test/point') + + Remove the last shape in the polygon shapefile: + + + >>> e = shapefile.Editor(shapefile="shapefiles/test/polygon.shp") + >>> e.delete(-1) + >>> e.save('shapefiles/test/polygon') + + ### Geometry and Record Balancing + + Because every shape must have a corresponding record it is critical that the + number of records equals the number of shapes to create a valid shapefile. To + help prevent accidental misalignment pyshp has an "auto balance" feature to + make sure when you add either a shape or a record the two sides of the + equation line up. This feature is NOT turned on by default. To activate it set + the attribute autoBalance to 1 (True): + + + >>> e.autoBalance = 1 + + You also have the option of manually calling the balance() method each time + you add a shape or a record to ensure the other side is up to date. When + balancing is used null shapes are created on the geometry side or a record + with a value of "NULL" for each field is created on the attribute side. + + The balancing option gives you flexibility in how you build the shapefile. + + Without auto balancing you can add geometry or records at anytime. You can + create all of the shapes and then create all of the records or vice versa. You + can use the balance method after creating a shape or record each time and make + updates later. If you do not use the balance method and forget to manually + balance the geometry and attributes the shapefile will be viewed as corrupt by + most shapefile software. + + With auto balancing you can add either shapes or geometry and update blank + entries on either side as needed. Even if you forget to update an entry the + shapefile will still be valid and handled correctly by most shapefile + software. + + ## Python \_\_geo_interface\_\_ + + The Python \_\_geo_interface\_\_ convention provides a data interchange interface + among geospatial Python libraries. The interface returns data as GeoJSON which gives you + nice compatibility with other libraries and tools including Shapely, Fiona, and PostGIS. + More information on the \_\_geo_interface\_\_ protocol can be found at: + [https://gist.github.com/sgillies/2217756](https://gist.github.com/sgillies/2217756). + More information on GeoJSON is available at [http://geojson.org](http://geojson.org). + + + >>> s = sf.shape(0) + >>> s.__geo_interface__["type"] + 'MultiPolygon' + + # Testing + + The testing framework is doctest, which are located in this file README.md. + In the same folder as README.md and shapefile.py, from the command line run + ``` + $ python shapefile.py + ``` + + Linux/Mac and similar platforms will need to run `$ dos2unix README.md` in order + correct line endings in README.md. + +Keywords: gis geospatial geographic shapefile shapefiles +Platform: UNKNOWN +Classifier: Programming Language :: Python +Classifier: Topic :: Scientific/Engineering :: GIS +Classifier: Topic :: Software Development :: Libraries +Classifier: Topic :: Software Development :: Libraries :: Python Modules diff --git a/README.md b/README.md index 5d1755b..380ffd2 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,32 @@ +# PyShp + +The Python Shapefile Library (pyshp) reads and writes ESRI Shapefiles in pure Python. + + + +[](https://travis-ci.org/GeospatialPython/pyshp) + ## Contents -[TOC] +[Overview](#overview) + +[Examples](#examples) +- [Reading Shapefiles](#reading-shapefiles) + - [Reading Shapefiles from File-Like Objects](#reading-shapefiles-from-file-like-objects) + - [Reading Geometry](#reading-geometry) + - [Reading Records](#reading-records) + - [Reading Geometry and Records Simultaneously](#reading-geometry-and-records-simultaneously) +- [Writing Shapefiles](#writing-shapefiles) + - [Setting the Shape Type](#setting-the-shape-type) + - [Geometry and Record Balancing](#geometry-and-record-balancing) + - [Adding Geometry](#adding-geometry) + - [Creating Attributes](#creating-attributes) + - [File Names](#file-names) + - [Saving to File-Like Objects](#saving-to-file-like-objects) +- [Editing Shapefiles](#editing-shapefiles) + - [Geometry and Record Balancing](#geometry-and-record-balancing) +- [Python \_\_geo_interface\_\_](#python-\_\_geo\_interface\_\_) +- [Testing](#testing) # Overview @@ -24,7 +50,7 @@ Pyshp is compatible with Python 2.4-3.x. This document provides examples for using pyshp to read and write shapefiles. However many more examples are continually added to the pyshp wiki on GitHub, the blog [http://GeospatialPython.com](http://GeospatialPython.com), -and by searching for pyshp on [http://gis.stackexchange.com](http://gis.stackexchange.com). +and by searching for pyshp on [https://gis.stackexchange.com](https://gis.stackexchange.com). Currently the sample census blockgroup shapefile referenced in the examples is available on the GitHub project site at [https://github.com/GeospatialPython/pyshp](https://github.com/GeospatialPython/pyshp). These @@ -45,13 +71,13 @@ Before doing anything you must import the library. >>> import shapefile The examples below will use a shapefile created from the U.S. Census Bureau -Blockgroups data set near San Francisco, CA and available in the github +Blockgroups data set near San Francisco, CA and available in the git repository of the pyshp GitHub site. ## Reading Shapefiles To read a shapefile create a new "Reader" object and pass it the name of an -existing shapefile. The shapefile format is acutally a collection of three +existing shapefile. The shapefile format is actually a collection of three files. You specify the base filename of the shapefile or the complete filename of any of the shapefile component files. @@ -92,7 +118,7 @@ it. ### Reading Geometry A shapefile's geometry is the collection of points or shapes made from -verticies and implied arcs representing physical locations. All types of +vertices and implied arcs representing physical locations. All types of shapefiles just store points. The metadata about the points determine how they are handled by software. @@ -259,14 +285,14 @@ To read a single record call the record() method with the record's index: ### Reading Geometry and Records Simultaneously -You way want to examine both the geometry and the attributes for a record at +You may want to examine both the geometry and the attributes for a record at the same time. The shapeRecord() and shapeRecords() method let you do just that. Calling the shapeRecords() method will return the geometry and attributes for all shapes as a list of ShapeRecord objects. Each ShapeRecord instance has a "shape" and "record" attribute. The shape attribute is a ShapeRecord object as -dicussed in the first section "Reading Geometry". The record attribute is a +discussed in the first section "Reading Geometry". The record attribute is a list of field values as demonstrated in the "Reading Records" section. @@ -287,7 +313,7 @@ Now let's read the first two points for that same record: 2 The shapeRecord() method reads a single shape/record pair at the specified index. -To get the 4th shape record from the blockgroups shapfile use the third index: +To get the 4th shape record from the blockgroups shapefile use the third index: >>> shapeRec = sf.shapeRecord(3) @@ -305,6 +331,7 @@ The blockgroup key and population count: There is also an iterShapeRecords() method to iterate through large files: + >>> shapeRecs = sf.iterShapeRecords() >>> for shapeRec in shapeRecs: ... # do something here @@ -321,7 +348,7 @@ PyShp can write just one of the component files such as the shp or dbf file without writing the others. So in addition to being a complete shapefile library, it can also be used as a basic dbf (xbase) library. Dbf files are a common database format which are often useful as a standalone simple database -format. And even shp files occasionaly have uses as a standalone format. Some +format. And even shp files occasionally have uses as a standalone format. Some web-based GIS systems use an user-uploaded shp file to specify an area of interest. Many precision agriculture chemical field sprayers also use the shp format as a control file for the sprayer system (usually in combination with @@ -348,7 +375,7 @@ the existing shape types are not sequential. There are three ways to set the shape type: * Set it when creating the class instance. * Set it by assigning a value to an existing class instance. - * Set it automatically to the type of the first shape by saving the shapefile. + * Set it automatically to the type of the first non-null shape by saving the shapefile. To manually set the shape type for a Writer object when creating the Writer: @@ -369,39 +396,23 @@ OR you can set it after the Writer is created: ### Geometry and Record Balancing Because every shape must have a corresponding record it is critical that the -number of records equals the number of shapes to create a valid shapefile. To -help prevent accidental misalignment the PSL has an "auto balance" feature to -make sure when you add either a shape or a record the two sides of the -equation line up. This feature is NOT turned on by default. To activate it set -the attribute autoBalance to 1 (True): +number of records equals the number of shapes to create a valid shapefile. You +must take care to add records and shapes in the same order so that the record +data lines up with the geometry data. For example: - >>> w.autoBalance = 1 - -You also have the option of manually calling the balance() method each time -you add a shape or a record to ensure the other side is up to date. When -balancing is used null shapes are created on the geometry side or a record -with a value of "NULL" for each field is created on the attribute side. - -The balancing option gives you flexibility in how you build the shapefile. - -Without auto balancing you can add geometry or records at anytime. You can -create all of the shapes and then create all of the records or vice versa. You -can use the balance method after creating a shape or record each time and make -updates later. If you do not use the balance method and forget to manually -balance the geometry and attributes the shapefile will be viewed as corrupt by -most shapefile software. - -With auto balanacing you can add either shapes or geometry and update blank -entries on either side as needed. Even if you forget to update an entry the -shapefile will still be valid and handled correctly by most shapefile -software. + >>> w.field("field1", "C") + >>> w.field("field2", "C") + >>> w.record("row", "one") + >>> w.record("row", "two") + >>> w.point(1, 1) + >>> w.point(2, 2) ### Adding Geometry Geometry is added using one of three methods: "null", "point", or "poly". The -"null" method is used for null shapes, "point" is used for point shapes, and -"poly" is used for everything else. +"null" method is used for null shapes, "point" is used for point shapes, "line" for lines, and +"poly" is used for polygons and everything else. **Adding a Point shape** @@ -409,8 +420,8 @@ Point shapes are added using the "point" method. A point is specified by an x, y, and optional z (elevation) and m (measure) value. - >>> w = shapefile.Writer() - + >>> w = shapefile.Writer(shapefile.POINTM) + >>> w.point(122, 37) # No elevation or measure values >>> w.shapes()[0].points @@ -420,21 +431,46 @@ y, and optional z (elevation) and m (measure) value. >>> w.shapes()[1].points [[118, 36, 4, 8]] + + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/point') -**Adding a Poly shape** +**Adding a Polygon shape** -"Poly" shapes can be either polygons or lines. Shapefile polygons must have at +Shapefile polygons must have at least 4 points and the last point must be the same as the first. PyShp -automatically enforces closed polygons. A line must have at least two points. -Because of the similarities between these two shape types they are created -using a single method called "poly". +automatically enforces closed polygons. >>> w = shapefile.Writer() - >>> w.poly(shapeType=3, parts=[[[122,37,4,9], [117,36,3,4]], [[115,32,8,8], + >>> w.poly(parts=[[[122,37,4,9], [117,36,3,4]], [[115,32,8,8], ... [118,20,6,4], [113,24]]]) + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/polygon') + +**Adding a Line shape** + +A line must have at least two points. +Because of the similarities between polygon and line types it is possible to create +a line shape using either the "line" or "poly" method. + + + >>> w = shapefile.Writer() + + >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]]) + >>> w.poly(parts=[[[1,3],[5,3]]], shapeType=shapefile.POLYLINE) + + >>> w.field('FIRST_FLD', 'C') + >>> w.field('SECOND_FLD', 'C') + + >>> w.save('shapefiles/test/polygon') + **Adding a Null shape** Because Null shape types (shape type 0) have no geometry the "null" method is @@ -454,57 +490,126 @@ The writer object's shapes list will now have one null shape: Creating attributes involves two steps. Step 1 is to create fields to contain attribute values and step 2 is to populate the fields with values for each -shape record. - -The following attempts to create a complete shapefile. The attribute and -field names are not very creative: - - - >>> w = shapefile.Writer(shapefile.POINT) - >>> w.point(1,1) - >>> w.point(3,1) - >>> w.point(4,3) - >>> w.point(2,2) - >>> w.field('FIRST_FLD') - >>> w.field('SECOND_FLD','C','40') - >>> w.record('First','Point') - >>> w.record('Second','Point') - >>> w.record('Third','Point') - >>> w.record('Fourth','Point') - >>> w.save('shapefiles/test/point') - - >>> w = shapefile.Writer(shapefile.POLYGON) - >>> w.poly(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]]) - >>> w.field('FIRST_FLD','C','40') - >>> w.field('SECOND_FLD','C','40') - >>> w.record('First','Polygon') - >>> w.save('shapefiles/test/polygon') - - >>> w = shapefile.Writer(shapefile.POLYLINE) - >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]]) - >>> w.poly(parts=[[[1,3],[5,3]]], shapeType=shapefile.POLYLINE) - >>> w.field('FIRST_FLD','C','40') - >>> w.field('SECOND_FLD','C','40') - >>> w.record('First','Line') - >>> w.record('Second','Line') - >>> w.save('shapefiles/test/line') +shape record. -You can also add attributes using keyword arguments where the keys are field -names. +There are several different field types, all of which support storing None values as NULL. +Text fields are created using the 'C' type, and the third 'size' argument can be customized to the expected +length of text values to save space: - >>> w = shapefile.Writer(shapefile.POLYLINE) - >>> w.line(parts=[[[1,5],[5,5],[5,1],[3,3],[1,1]]]) - >>> w.field('FIRST_FLD','C','40') - >>> w.field('SECOND_FLD','C','40') - >>> w.record(FIRST_FLD='First', SECOND_FLD='Line') - >>> w.save('shapefiles/test/line') + + >>> w = shapefile.Writer() + >>> w.field('TEXT', 'C') + >>> w.field('SHORT_TEXT', 'C', size=5) + >>> w.field('LONG_TEXT', 'C', size=250) + >>> w.null() + >>> w.record('Hello', 'World', 'World'*50) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == ['Hello', 'World', 'World'*50] + +Date fields are created using the 'D' type, and can be created using either +date objects, lists, or a YYYYMMDD formatted string. +Field length or decimal have no impact on this type: + + + >>> from datetime import date + >>> w = shapefile.Writer() + >>> w.field('DATE', 'D') + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.record(date(1998,1,30)) + >>> w.record([1998,1,30]) + >>> w.record('19980130') + >>> w.record(None) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == [date(1998,1,30)] + >>> assert r.record(1) == [date(1998,1,30)] + >>> assert r.record(2) == [date(1998,1,30)] + >>> assert r.record(3) == [None] + +Numeric fields are created using the 'N' type (or the 'F' type, which is exactly the same). +By default the fourth decimal argument is set to zero, essentially creating an integer field. +To store floats you must set the decimal argument to the precision of your choice. +To store very large numbers you must increase the field length size to the total number of digits +(including comma and minus). + + + >>> w = shapefile.Writer() + >>> w.field('INT', 'N') + >>> w.field('LOWPREC', 'N', decimal=2) + >>> w.field('MEDPREC', 'N', decimal=10) + >>> w.field('HIGHPREC', 'N', decimal=30) + >>> w.field('FTYPE', 'F', decimal=10) + >>> w.field('LARGENR', 'N', 101) + >>> nr = 1.3217328 + >>> w.null() + >>> w.null() + >>> w.record(INT=int(nr), LOWPREC=nr, MEDPREC=nr, HIGHPREC=-3.2302e-25, FTYPE=nr, LARGENR=int(nr)*10**100) + >>> w.record(None, None, None, None, None, None) + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> assert r.record(0) == [1, 1.32, 1.3217328, -3.2302e-25, 1.3217328, 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000] + >>> assert r.record(1) == [None, None, None, None, None, None] + + +Finally, we can create boolean fields by setting the type to 'L'. +This field can take True or False values, or 1 (True) or 0 (False). +None is interpreted as missing. + + + >>> w = shapefile.Writer() + >>> w.field('BOOLEAN', 'L') + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.null() + >>> w.record(True) + >>> w.record(1) + >>> w.record(False) + >>> w.record(0) + >>> w.record(None) + >>> w.record("Nonesense") + >>> w.save('shapefiles/test/dtype') + + >>> r = shapefile.Reader('shapefiles/test/dtype') + >>> r.record(0) + [True] + >>> r.record(1) + [True] + >>> r.record(2) + [False] + >>> r.record(3) + [False] + >>> r.record(4) + [None] + >>> r.record(5) + [None] + +You can also add attributes using keyword arguments where the keys are field names. + + + >>> w = shapefile.Writer() + >>> w.field('FIRST_FLD','C','40') + >>> w.field('SECOND_FLD','C','40') + >>> w.record('First', 'Line') + >>> w.record(FIRST_FLD='First', SECOND_FLD='Line') + >>> assert w.records[0] == w.records[1] + + ### File Names -File extensions are optional when reading or writing shapfiles. If you specify +File extensions are optional when reading or writing shapefiles. If you specify them PyShp ignores them anyway. When you save files you can specify a base -file name that is used for all three file types. Or you can specify a nmae for +file name that is used for all three file types. Or you can specify a name for one or more file types. In that case, any file types not assigned will not save and only file types with file names will be saved. If you do not specify any file names (i.e. save()), then a unique file name is generated with the @@ -544,7 +649,7 @@ and write out a new shapefile with the same or different name.* Let's add shapes to existing shapefiles: -Add a point to a point shapefile +Add a point to a point shapefile: >>> e = shapefile.Editor(shapefile="shapefiles/test/point.shp") @@ -569,29 +674,60 @@ Add a new polygon to a polygon shapefile: >>> e.save('shapefiles/test/polygon') Remove the first point in each shapefile - for a point shapefile that is the -first shape and record" +first shape and record": >>> e = shapefile.Editor(shapefile="shapefiles/test/point.shp") >>> e.delete(0) >>> e.save('shapefiles/test/point') -Remove the last shape in the polygon shapefile. +Remove the last shape in the polygon shapefile: >>> e = shapefile.Editor(shapefile="shapefiles/test/polygon.shp") >>> e.delete(-1) >>> e.save('shapefiles/test/polygon') + +### Geometry and Record Balancing + +Because every shape must have a corresponding record it is critical that the +number of records equals the number of shapes to create a valid shapefile. To +help prevent accidental misalignment pyshp has an "auto balance" feature to +make sure when you add either a shape or a record the two sides of the +equation line up. This feature is NOT turned on by default. To activate it set +the attribute autoBalance to 1 (True): + + + >>> e.autoBalance = 1 + +You also have the option of manually calling the balance() method each time +you add a shape or a record to ensure the other side is up to date. When +balancing is used null shapes are created on the geometry side or a record +with a value of "NULL" for each field is created on the attribute side. + +The balancing option gives you flexibility in how you build the shapefile. + +Without auto balancing you can add geometry or records at anytime. You can +create all of the shapes and then create all of the records or vice versa. You +can use the balance method after creating a shape or record each time and make +updates later. If you do not use the balance method and forget to manually +balance the geometry and attributes the shapefile will be viewed as corrupt by +most shapefile software. + +With auto balancing you can add either shapes or geometry and update blank +entries on either side as needed. Even if you forget to update an entry the +shapefile will still be valid and handled correctly by most shapefile +software. ## Python \_\_geo_interface\_\_ The Python \_\_geo_interface\_\_ convention provides a data interchange interface among geospatial Python libraries. The interface returns data as GeoJSON which gives you -nice compatability with other libraries and tools including Shapely, Fiona, and PostGIS. -More information on the \_\_geo_interface\_\_ protocol can be found at: [https://gist.g -ithub.com/sgillies/2217756](https://gist.github.com/sgillies/2217756). More -information on GeoJSON is available at -[http://geojson.org](http://geojson.org). +nice compatibility with other libraries and tools including Shapely, Fiona, and PostGIS. +More information on the \_\_geo_interface\_\_ protocol can be found at: +[https://gist.github.com/sgillies/2217756](https://gist.github.com/sgillies/2217756). +More information on GeoJSON is available at [http://geojson.org](http://geojson.org). + >>> s = sf.shape(0) >>> s.__geo_interface__["type"] diff --git a/changelog.txt b/changelog.txt index 65373e9..9ff0a00 100644 --- a/changelog.txt +++ b/changelog.txt @@ -1,3 +1,15 @@ +VERSION 1.2.11 + +2017-04-29 Karim Bahgat <[email protected]> + * Fixed bugs when reading and writing empty shapefiles. + * Fixed bug when writing null geometry. + * Fixed misc data type errors. + * Fixed error when reading files with wrong record length. + * Use max field precision when saving decimal numbers. + * Improved shapetype detection. + * Expanded docs on data types. + * General doc additions and travis icon. + VERSION 1.2.10 2016-09-24 Karim Bahgat <[email protected]> diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 0000000..861a9f5 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,5 @@ +[egg_info] +tag_build = +tag_date = 0 +tag_svn_revision = 0 + diff --git a/setup.py b/setup.py index 199ef22..9acd4f7 100644 --- a/setup.py +++ b/setup.py @@ -1,7 +1,7 @@ from setuptools import setup setup(name='pyshp', - version='1.2.10', + version='1.2.11', description='Pure Python read/write support for ESRI Shapefile format', long_description=open('README.md').read(), author='Joel Lawhead', diff --git a/shapefile.py b/shapefile.py index 44201ab..c3eb89e 100644 --- a/shapefile.py +++ b/shapefile.py @@ -2,14 +2,12 @@ shapefile.py Provides read and write support for ESRI Shapefiles. author: jlawhead<at>geospatialpython.com -date: 2016/09/24 -version: 1.2.10 -Compatible with Python versions 2.4-3.x -version changelog: -- Bump version to fix pip install issue. +date: 2017/04/29 +version: 1.2.11 +Compatible with Python versions 2.7-3.x """ -__version__ = "1.2.10" +__version__ = "1.2.11" from struct import pack, unpack, calcsize, error, Struct import os @@ -18,6 +16,7 @@ import time import array import tempfile import itertools +from datetime import date # # Constants for shape types @@ -36,6 +35,8 @@ POLYGONM = 25 MULTIPOINTM = 28 MULTIPATCH = 31 +MISSING = [None,''] + PYTHON3 = sys.version_info[0] == 3 if PYTHON3: @@ -105,7 +106,7 @@ def signed_area(coords): return sum(xs[i]*(ys[i+1]-ys[i-1]) for i in range(1, len(coords)))/2.0 class _Shape: - def __init__(self, shapeType=None): + def __init__(self, shapeType=NULL): """Stores the geometry of the different shape types specified in the Shapefile spec. Shape types are usually point, polyline, or polygons. Every shape type @@ -117,6 +118,7 @@ class _Shape: list of shapes.""" self.shapeType = shapeType self.points = [] + self.parts = [] @property def __geo_interface__(self): @@ -256,23 +258,24 @@ class Reader: def load(self, shapefile=None): """Opens a shapefile from a filename or file-like object. Normally this method would be called by the - constructor with the file object or file name as an - argument.""" + constructor with the file name as an argument.""" if shapefile: (shapeName, ext) = os.path.splitext(shapefile) self.shapeName = shapeName try: self.shp = open("%s.shp" % shapeName, "rb") except IOError: - raise ShapefileException("Unable to open %s.shp" % shapeName) + pass try: self.shx = open("%s.shx" % shapeName, "rb") except IOError: - raise ShapefileException("Unable to open %s.shx" % shapeName) + pass try: self.dbf = open("%s.dbf" % shapeName, "rb") except IOError: - raise ShapefileException("Unable to open %s.dbf" % shapeName) + pass + if not (self.shp and self.dbf): + raise ShapefileException("Unable to open %s.dbf or %s.shp." % (shapeName, shapeName) ) if self.shp: self.__shpHeader() if self.dbf: @@ -439,23 +442,16 @@ class Reader: while shp.tell() < self.shpLength: yield self.__shape() - def __dbfHeaderLength(self): - """Retrieves the header length of a dbf file header.""" - if not self.__dbfHdrLength: - if not self.dbf: - raise ShapefileException("Shapefile Reader requires a shapefile or file-like object. (no dbf file found)") - dbf = self.dbf - (self.numRecords, self.__dbfHdrLength) = \ - unpack("<xxxxLH22x", dbf.read(32)) - return self.__dbfHdrLength - def __dbfHeader(self): """Reads a dbf header. Xbase-related code borrows heavily from ActiveState Python Cookbook Recipe 362715 by Raymond Hettinger""" if not self.dbf: raise ShapefileException("Shapefile Reader requires a shapefile or file-like object. (no dbf file found)") dbf = self.dbf - headerLength = self.__dbfHeaderLength() - numFields = (headerLength - 33) // 32 + # read relevant header parts + self.numRecords, self.__dbfHdrLength, self.__recordLength = \ + unpack("<xxxxLHH20x", dbf.read(32)) + # read fields + numFields = (self.__dbfHdrLength - 33) // 32 for field in range(numFields): fieldDesc = list(unpack("<11sc4xBB14x", dbf.read(32))) name = 0 @@ -478,10 +474,15 @@ class Reader: def __recordFmt(self): """Calculates the format and size of a .dbf record.""" - if not self.numRecords: + if self.numRecords is None: self.__dbfHeader() fmt = ''.join(['%ds' % fieldinfo[2] for fieldinfo in self.fields]) fmtSize = calcsize(fmt) + # total size of fields should add up to recordlength from the header + while fmtSize < self.__recordLength: + # if not, pad byte until reaches recordlength + fmt += "x" + fmtSize += 1 return (fmt, fmtSize) def __record(self): @@ -492,14 +493,11 @@ class Reader: # deleted record return None record = [] - for (name, typ, size, deci), value in zip(self.fields, - recordContents): + for (name, typ, size, deci), value in zip(self.fields, recordContents): if name == 'DeletionFlag': continue - elif not value.strip(): - record.append(value) - continue - elif typ == "N": + elif typ in ("N","F"): + # numeric or float: number stored as a string, right justified, and padded with blanks to the width of the field. value = value.replace(b('\0'), b('')).strip() value = value.replace(b('*'), b('')) # QGIS NULL is all '*' chars if value == b(''): @@ -516,19 +514,29 @@ class Reader: except ValueError: #not parseable as int, set to None value = None - elif typ == b('D'): + elif typ == 'D': + # date: 8 bytes - date stored as a string in the format YYYYMMDD. if value.count(b('0')) == len(value): # QGIS NULL is all '0' chars value = None else: try: y, m, d = int(value[:4]), int(value[4:6]), int(value[6:8]) - value = [y, m, d] + value = date(y, m, d) except: value = value.strip() - elif typ == b('L'): - value = (value in b('YyTt') and b('T')) or \ - (value in b('NnFf') and b('F')) or b('?') + elif typ == 'L': + # logical: 1 byte - initialized to 0x20 (space) otherwise T or F. + if value == b(" "): + value = None # space means missing or not yet set + else: + if value in b('YyTt1'): + value = True + elif value in b('NnFf0'): + value = False + else: + value = None # unknown value is set to missing else: + # anything else is forced to string/unicode value = u(value) value = value.strip() record.append(value) @@ -537,21 +545,21 @@ class Reader: def record(self, i=0): """Returns a specific dbf record based on the supplied index.""" f = self.__getFileObj(self.dbf) - if not self.numRecords: + if self.numRecords is None: self.__dbfHeader() i = self.__restrictIndex(i) recSize = self.__recStruct.size f.seek(0) - f.seek(self.__dbfHeaderLength() + (i * recSize)) + f.seek(self.__dbfHdrLength + (i * recSize)) return self.__record() def records(self): """Returns all records in a dbf file.""" - if not self.numRecords: + if self.numRecords is None: self.__dbfHeader() records = [] f = self.__getFileObj(self.dbf) - f.seek(self.__dbfHeaderLength()) + f.seek(self.__dbfHdrLength) for i in range(self.numRecords): r = self.__record() if r: @@ -561,10 +569,10 @@ class Reader: def iterRecords(self): """Serves up records in a dbf file as an iterator. Useful for large shapefiles or dbf files.""" - if not self.numRecords: + if self.numRecords is None: self.__dbfHeader() f = self.__getFileObj(self.dbf) - f.seek(self.__dbfHeaderLength()) + f.seek(self.__dbfHdrLength) for i in xrange(self.numRecords): r = self.__record() if r: @@ -659,7 +667,7 @@ class Writer: # z array size += 8 * nPoints # Calc m extremes and values - if self.shapeType in (23,25,31): + if self.shapeType in (15,23,25,31): # m extremes size += 16 # m array @@ -677,19 +685,19 @@ class Writer: size //= 2 return size - def __bbox(self, shapes, shapeTypes=[]): + def __bbox(self, shapes): x = [] y = [] for s in shapes: - shapeType = self.shapeType - if shapeTypes: - shapeType = shapeTypes[shapes.index(s)] - px, py = list(zip(*s.points))[:2] - x.extend(px) - y.extend(py) + if len(s.points) > 0: + px, py = list(zip(*s.points))[:2] + x.extend(px) + y.extend(py) + if len(x) == 0: + return [0] * 4 return [min(x), min(y), max(x), max(y)] - def __zbox(self, shapes, shapeTypes=[]): + def __zbox(self, shapes): z = [] for s in shapes: try: @@ -700,14 +708,15 @@ class Writer: if not z: z.append(0) return [min(z), max(z)] - def __mbox(self, shapes, shapeTypes=[]): - m = [0] + def __mbox(self, shapes): + m = [] for s in shapes: try: for p in s.points: m.append(p[3]) except IndexError: pass + if not m: m.append(0) return [min(m), max(m)] def bbox(self): @@ -765,7 +774,7 @@ class Writer: year -= 1900 # Remove deletion flag placeholder from fields for field in self.fields: - if field[0].startswith("Deletion"): + if str(field[0]).startswith("Deletion"): self.fields.remove(field) numRecs = len(self.records) numFields = len(self.fields) @@ -799,8 +808,6 @@ class Writer: recNum += 1 start = f.tell() # Shape Type - if self.shapeType != 31: - s.shapeType = self.shapeType f.write(pack("<i", s.shapeType)) # All shape types capable of having a bounding box if s.shapeType in (3,5,8,13,15,18,23,25,28,31): @@ -846,14 +853,14 @@ class Writer: # Write m extremes and values if s.shapeType in (13,15,18,23,25,28,31): try: - if hasattr(s,"m"): + if hasattr(s,"m") and None not in s.m: f.write(pack("<%sd" % len(s.m), *s.m)) else: f.write(pack("<2d", *self.__mbox([s]))) except error: raise ShapefileException("Failed to write measure extremes for record %s. Expected floats" % recNum) try: - [f.write(pack("<d", p[3])) for p in s.points] + [f.write(pack("<d", len(p) > 3 and p[3] or 0)) for p in s.points] except error: raise ShapefileException("Failed to write measure values for record %s. Expected floats" % recNum) # Write a single point @@ -917,14 +924,41 @@ class Writer: for record in self.records: if not self.fields[0][0].startswith("Deletion"): f.write(b(' ')) # deletion flag - for (fieldName, fieldType, size, dec), value in zip(self.fields, record): + for (fieldName, fieldType, size, deci), value in zip(self.fields, record): fieldType = fieldType.upper() size = int(size) - if fieldType.upper() == "N": - value = str(value).rjust(size) + if fieldType in ("N","F"): + # numeric or float: number stored as a string, right justified, and padded with blanks to the width of the field. + if value in MISSING: + value = str("*"*size) # QGIS NULL + elif not deci: + value = format(value, "d")[:size].rjust(size) # caps the size if exceeds the field size + else: + value = format(value, ".%sf"%deci)[:size].rjust(size) # caps the size if exceeds the field size + elif fieldType == "D": + # date: 8 bytes - date stored as a string in the format YYYYMMDD. + if isinstance(value, date): + value = value.strftime("%Y%m%d") + elif isinstance(value, list) and len(value) == 3: + value = date(*value).strftime("%Y%m%d") + elif value in MISSING: + value = b('0') * 8 # QGIS NULL for date type + elif isinstance(value, str) and len(value) == 8: + pass # value is already a date string + else: + raise ShapefileException("Date values must be either a datetime.date object, a list, a YYYYMMDD string, or a missing value.") elif fieldType == 'L': - value = str(value)[0].upper() + # logical: 1 byte - initialized to 0x20 (space) otherwise T or F. + if value in MISSING: + value = b(' ') # missing is set to space + elif value in [True,1]: + value = b("T") + elif value in [False,0]: + value = b("F") + else: + value = b(' ') # unknown is set to space else: + # anything else is forced to string value = str(value)[:size].ljust(size) if len(value) != size: raise ShapefileException( @@ -937,9 +971,9 @@ class Writer: """Creates a null shape.""" self._shapes.append(_Shape(NULL)) - def point(self, x, y, z=0, m=0): + def point(self, x, y, z=0, m=0, shapeType=POINT): """Creates a point shape.""" - pointShape = _Shape(self.shapeType) + pointShape = _Shape(shapeType) pointShape.points.append([x, y, z, m]) self._shapes.append(pointShape) @@ -982,6 +1016,12 @@ class Writer: def field(self, name, fieldType="C", size="50", decimal=0): """Adds a dbf field descriptor to the shapefile.""" + if fieldType == "D": + size = "8" + decimal = 0 + elif fieldType == "L": + size = "1" + decimal = 0 self.fields.append((name, fieldType, size, decimal)) def record(self, *recordList, **recordDict): @@ -997,7 +1037,7 @@ class Writer: # Compensate for deletion flag if self.fields[0][0].startswith("Deletion"): fieldCount -= 1 if recordList: - [record.append(recordList[i]) for i in range(fieldCount)] + record = [recordList[i] for i in range(fieldCount)] elif recordDict: for field in self.fields: if field[0] in recordDict: @@ -1006,8 +1046,10 @@ class Writer: record.append("") else: record.append(val) - if record: - self.records.append(record) + else: + # Blank fields for empty record + record = ["" for i in range(fieldCount)] + self.records.append(record) def shape(self, i): return self._shapes[i] @@ -1020,8 +1062,9 @@ class Writer: """Save an shp file.""" if not hasattr(target, "write"): target = os.path.splitext(target)[0] + '.shp' - if not self.shapeType: - self.shapeType = self._shapes[0].shapeType + if self.shapeType is None: + # autoset file type to first non-null geometry + self.shapeType = next((s.shapeType for s in self._shapes if s.shapeType), NULL) self.shp = self.__getFileObj(target) self.__shapefileHeader(self.shp, headerType='shp') self.__shpRecords() @@ -1030,8 +1073,9 @@ class Writer: """Save an shx file.""" if not hasattr(target, "write"): target = os.path.splitext(target)[0] + '.shx' - if not self.shapeType: - self.shapeType = self._shapes[0].shapeType + if self.shapeType is None: + # autoset file type to first non-null geometry + self.shapeType = next((s.shapeType for s in self._shapes if s.shapeType != NULL), NULL) self.shx = self.__getFileObj(target) self.__shapefileHeader(self.shx, headerType='shx') self.__shxRecords() @@ -1074,6 +1118,7 @@ class Writer: self.dbf.close() if generated: return target + class Editor(Writer): def __init__(self, shapefile=None, shapeType=POINT, autoBalance=1): self.autoBalance = autoBalance @@ -1097,16 +1142,19 @@ class Editor(Writer): """Deletes the specified part of any shape by specifying a shape number, part number, or point number.""" # shape, part, point - if shape and part and point: + shape_param_exists = shape is not None + part_param_exists = part is not None + point_param_exists = point is not None + if shape_param_exists and part_param_exists and point_param_exists: del self._shapes[shape][part][point] # shape, part - elif shape and part and not point: + elif shape_param_exists and part_param_exists and not point_param_exists: del self._shapes[shape][part] # shape - elif shape and not part and not point: + elif shape_param_exists and not part_param_exists and not point_param_exists: del self._shapes[shape] # point - elif not shape and not part and point: + elif not shape_param_exists and not part_param_exists and point_param_exists: for s in self._shapes: if s.shapeType == 1: del self._shapes[point] @@ -1114,11 +1162,11 @@ class Editor(Writer): for part in s.parts: del s[part][point] # part, point - elif not shape and part and point: + elif not shape_param_exists and part_param_exists and point_param_exists: for s in self._shapes: del s[part][point] # part - elif not shape and part and not point: + elif not shape_param_exists and part_param_exists and not point_param_exists: for s in self._shapes: del s[part] @@ -1192,11 +1240,17 @@ class Editor(Writer): fieldName.replace(' ', '_') # Begin Testing -def test(): +def test(**kwargs): import doctest doctest.NORMALIZE_WHITESPACE = 1 - doctest.testfile("README.md", verbose=1) - + verbosity = kwargs.get('verbose', 0) + if verbosity == 0: + print('Running doctests...') + failure_count, test_count = doctest.testfile("README.md", verbose=verbosity) + if verbosity == 0 and failure_count == 0: + print('All test passed successfully') + return failure_count + if __name__ == "__main__": """ Doctests are contained in the file 'README.md'. This library was originally developed @@ -1204,4 +1258,5 @@ if __name__ == "__main__": testing libraries but for now unit testing is done using what's available in 2.3. """ - test() + failure_count = test() + sys.exit(failure_count) diff --git a/shapefiles/test/dtype.dbf b/shapefiles/test/dtype.dbf new file mode 100644 index 0000000..8226836 Binary files /dev/null and b/shapefiles/test/dtype.dbf differ diff --git a/shapefiles/test/dtype.shp b/shapefiles/test/dtype.shp new file mode 100644 index 0000000..d5fe49e Binary files /dev/null and b/shapefiles/test/dtype.shp differ diff --git a/shapefiles/test/dtype.shx b/shapefiles/test/dtype.shx new file mode 100644 index 0000000..7f9eccc Binary files /dev/null and b/shapefiles/test/dtype.shx differ diff --git a/shapefiles/test/line.dbf b/shapefiles/test/line.dbf new file mode 100644 index 0000000..72ac67f Binary files /dev/null and b/shapefiles/test/line.dbf differ diff --git a/shapefiles/test/line.shp b/shapefiles/test/line.shp new file mode 100644 index 0000000..3b930c6 Binary files /dev/null and b/shapefiles/test/line.shp differ diff --git a/shapefiles/test/line.shx b/shapefiles/test/line.shx new file mode 100644 index 0000000..b1c909d Binary files /dev/null and b/shapefiles/test/line.shx differ diff --git a/shapefiles/test/point.dbf b/shapefiles/test/point.dbf new file mode 100644 index 0000000..c77d097 Binary files /dev/null and b/shapefiles/test/point.dbf differ diff --git a/shapefiles/test/point.shp b/shapefiles/test/point.shp new file mode 100644 index 0000000..8ceb215 Binary files /dev/null and b/shapefiles/test/point.shp differ diff --git a/shapefiles/test/point.shx b/shapefiles/test/point.shx new file mode 100644 index 0000000..8605824 Binary files /dev/null and b/shapefiles/test/point.shx differ diff --git a/shapefiles/test/polygon.dbf b/shapefiles/test/polygon.dbf new file mode 100644 index 0000000..184088d Binary files /dev/null and b/shapefiles/test/polygon.dbf differ diff --git a/shapefiles/test/polygon.shp b/shapefiles/test/polygon.shp new file mode 100644 index 0000000..9d36cf0 Binary files /dev/null and b/shapefiles/test/polygon.shp differ diff --git a/shapefiles/test/polygon.shx b/shapefiles/test/polygon.shx new file mode 100644 index 0000000..ea34f73 Binary files /dev/null and b/shapefiles/test/polygon.shx differ -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-grass/pyshp.git _______________________________________________ Pkg-grass-devel mailing list [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-grass-devel
