This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch branch-1.5.2 in repository https://gitbox.apache.org/repos/asf/sedona.git
commit 8bd39cd7efb0760cf3673214ce4025ce0b5d8dfb Author: Jia Yu <[email protected]> AuthorDate: Sat Apr 27 22:54:44 2024 -0700 Revert "[DOCS] Enable markdownlint rule `MD031` (#1331)" This reverts commit 9eca25445de77a17ff9e814771fd8a6215d58c0c. --- .github/linters/.markdown-lint.yml | 3 + README.md | 2 - docs/api/flink/Constructor.md | 4 -- docs/api/flink/Function.md | 34 ----------- docs/api/snowflake/vector-data/Constructor.md | 1 - docs/api/snowflake/vector-data/Function.md | 70 ---------------------- docs/api/sql/Constructor.md | 1 - docs/api/sql/DataFrameAPI.md | 2 - docs/api/sql/Function.md | 31 ---------- docs/api/sql/Optimizer.md | 7 --- docs/api/sql/Overview.md | 5 -- docs/api/sql/Parameter.md | 5 -- docs/api/sql/Raster-loader.md | 1 - docs/api/sql/Raster-operators.md | 17 ------ docs/api/sql/Raster-visualizer.md | 1 - docs/api/sql/Visualization_SedonaKepler.md | 2 - docs/api/sql/Visualization_SedonaPyDeck.md | 3 - docs/api/viz/sql.md | 5 -- docs/community/publish.md | 5 -- docs/community/release-manager.md | 9 --- docs/community/snapshot.md | 2 - docs/setup/databricks.md | 7 --- docs/setup/docker.md | 1 - docs/setup/flink/install-scala.md | 1 - docs/setup/install-scala.md | 5 -- docs/setup/release-notes.md | 5 -- .../Advanced-Tutorial-Tune-your-Application.md | 4 -- docs/tutorial/flink/sql.md | 4 -- docs/tutorial/geopandas-shapely.md | 1 - docs/tutorial/rdd.md | 7 --- docs/tutorial/snowflake/sql.md | 2 - docs/tutorial/sql.md | 2 - docs/tutorial/zeppelin.md | 2 - 33 files changed, 3 insertions(+), 248 deletions(-) diff --git a/.github/linters/.markdown-lint.yml b/.github/linters/.markdown-lint.yml index a6a1f57f6..c8583024a 100644 --- a/.github/linters/.markdown-lint.yml +++ b/.github/linters/.markdown-lint.yml @@ -21,6 +21,9 @@ MD026: false # ol-prefix - Ordered list item prefix MD029: false +# blanks-around-fences - Fenced code blocks should be surrounded by blank lines +MD031: false + # blanks-around-lists - Lists should be surrounded by blank lines MD032: false diff --git a/README.md b/README.md index b4f659d8d..642d7cd2b 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,6 @@ This example loads NYC taxi trip records and taxi zone information stored as .CS taxidf = sedona.read.format('csv').option("header","true").option("delimiter", ",").load("s3a://your-directory/data/nyc-taxi-data.csv") taxidf = taxidf.selectExpr('ST_Point(CAST(Start_Lon AS Decimal(24,20)), CAST(Start_Lat AS Decimal(24,20))) AS pickup', 'Trip_Pickup_DateTime', 'Payment_Type', 'Fare_Amt') ``` - ```python zoneDf = sedona.read.format('csv').option("delimiter", ",").load("s3a://your-directory/data/TIGER2018_ZCTA5.csv") zoneDf = zoneDf.selectExpr('ST_GeomFromWKT(_c0) as zone', '_c1 as zipcode') @@ -123,7 +122,6 @@ We provide a Docker image for Apache Sedona with Python JupyterLab and a single- ``` pip install apache-sedona ``` - * To compile the source code, please refer to [Sedona website](https://sedona.apache.org/latest-snapshot/setup/compile/) * Modules in the source code diff --git a/docs/api/flink/Constructor.md b/docs/api/flink/Constructor.md index 428ad5957..0d071ba8a 100644 --- a/docs/api/flink/Constructor.md +++ b/docs/api/flink/Constructor.md @@ -8,7 +8,6 @@ Format: Since: `v1.5.0` Example: - ```sql SELECT ST_AsText(ST_GeomFromEWKT('SRID=4269;POINT(40.7128 -74.0060)')) ``` @@ -228,7 +227,6 @@ Format: Since: `v1.2.1` Example: - ```sql SELECT ST_GeomFromWKB(polygontable._c0) AS polygonshape FROM polygontable @@ -244,7 +242,6 @@ Format: Since: `v1.2.0` Example: - ```sql SELECT ST_GeomFromWKT('POINT(40.7128 -74.0060)') ``` @@ -264,7 +261,6 @@ Format: `ST_LineFromText (Text: String)` Since: `v1.2.1` Example: - ```sql SELECT ST_LineFromText('Linestring(1 2, 3 4)') ``` diff --git a/docs/api/flink/Function.md b/docs/api/flink/Function.md index e5e3287ae..48978c023 100644 --- a/docs/api/flink/Function.md +++ b/docs/api/flink/Function.md @@ -48,7 +48,6 @@ SELECT ST_3DDistance(ST_GeomFromText("POINT Z (0 0 -5)"), ``` Output: - ``` 1.7320508075688772 ``` @@ -66,7 +65,6 @@ Format: Since: `v1.3.0` Example: - ```sql SELECT ST_AddPoint(ST_GeomFromText("LINESTRING(0 0, 1 1, 1 0)"), ST_GeomFromText("Point(21 52)"), 1) @@ -74,7 +72,6 @@ SELECT ST_AddPoint(ST_GeomFromText("Linestring(0 0, 1 1, 1 0)"), ST_GeomFromText ``` Output: - ``` LINESTRING(0 0, 21 52, 1 1, 1 0) LINESTRING(0 0, 1 1, 1 0, 21 52) @@ -225,7 +222,6 @@ SELECT ST_Area(ST_GeomFromText("POLYGON(0 0, 0 10, 10 10, 0 10, 0 0)")) ``` Output: - ``` 10 ``` @@ -313,7 +309,6 @@ Format: `ST_AsEWKT (A: Geometry)` Since: `v1.2.1` Example: - ```sql SELECT ST_AsEWKT(ST_SetSrid(ST_GeomFromWKT('POLYGON((0 0,0 1,1 1,1 0,0 0))'), 4326)) ``` @@ -357,7 +352,6 @@ Format: `ST_AsGeoJSON (A: Geometry)` Since: `v1.3.0` Example: - ```sql SELECT ST_AsGeoJSON(ST_GeomFromWKT('POLYGON((1 1, 8 1, 8 8, 1 8, 1 1))')) ``` @@ -513,7 +507,6 @@ Format: `ST_BoundingDiagonal(geom: Geometry)` Since: `v1.5.0` Example: - ```sql SELECT ST_BoundingDiagonal(ST_GeomFromWKT(geom)) ``` @@ -606,7 +599,6 @@ Format: `ST_Centroid (A: Geometry)` Since: `v1.5.0` Example: - ```sql SELECT ST_Centroid(ST_GeomFromWKT('MULTIPOINT(-1 0, -1 2, 7 8, 9 8, 10 6)')) ``` @@ -627,7 +619,6 @@ Format: `ST_ClosestPoint(g1: Geometry, g2: Geometry)` Since: `v1.5.0` Example: - ```sql SELECT ST_AsText( ST_ClosestPoint(g1, g2)) As ptwkt; ``` @@ -966,7 +957,6 @@ Format: `ST_Dump(geom: Geometry)` Since: `v1.5.0` Example: - ```sql SELECT ST_Dump(ST_GeomFromText('MULTIPOINT ((10 40), (40 30), (20 20), (30 10))')) ``` @@ -1221,13 +1211,11 @@ Default parameters: `tolerance: 1e-6, maxIter: 1000, failIfNotConverged: false` Since: `v1.4.1` Example: - ```sql SELECT ST_GeometricMedian(ST_GeomFromWKT('MULTIPOINT((0 0), (1 1), (2 2), (200 200))')) ``` Output: - ``` POINT (1.9761550281255005 1.9761550281255005) ``` @@ -1285,13 +1273,11 @@ Format: `ST_H3CellDistance(cell1: Long, cell2: Long)` Since: `v1.5.0` Example: - ```sql select ST_H3CellDistance(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[1], ST_H3CellIDs(ST_GeomFromWKT('POINT(1.23 1.59)'), 8, true)[1]) ``` Output: - ``` +----+----------------------+ | op | EXPR$0 | @@ -1334,13 +1320,11 @@ Format: `ST_H3CellIDs(geom: geometry, level: Int, fullCover: true)` Since: `v1.5.0` Example: - ```sql SELECT ST_H3CellIDs(ST_GeomFromText('LINESTRING(1 3 4, 5 6 7)'), 6, true) ``` Output: - ``` +----+--------------------------------+ | op | EXPR$0 | @@ -1363,13 +1347,11 @@ Format: `ST_H3KRing(cell: Long, k: Int, exactRing: Boolean)` Since: `v1.5.0` Example: - ```sql select ST_H3KRing(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[1], 1, false), ST_H3KRing(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[1], 1, true) ``` Output: - ``` +----+--------------------------------+--------------------------------+ | op | EXPR$0 | EXPR$1 | @@ -1389,13 +1371,11 @@ Format: `ST_H3ToGeom(cells: Array[Long])` Since: `v1.5.0` Example: - ```sql SELECT ST_H3ToGeom(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[0], 1, true)) ``` Output: - ``` |st_h3togeom(st_h3cellids(st_geomfromwkt(POINT(1 2), 0), 8, true)) | +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -1614,11 +1594,9 @@ Introduction: Test if a geometry is well-formed. The function can be invoked wit - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValid (A: Geometry) ``` - ``` ST_IsValid (A: Geometry, flag: Integer) ``` @@ -1645,11 +1623,9 @@ Introduction: Returns text stating if the geometry is valid. If not, it provides - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValidReason (A: Geometry) ``` - ``` ST_IsValidReason (A: Geometry, flag: Integer) ``` @@ -1768,7 +1744,6 @@ SELECT ST_LineInterpolatePoint(ST_GeomFromWKT('LINESTRING(25 50, 100 125, 150 19 ``` Output: - ``` POINT (51.5974135047432 76.5974135047432) ``` @@ -1789,7 +1764,6 @@ SELECT ST_LineLocatePoint(ST_GeomFromWKT('LINESTRING(0 0, 1 1, 2 2)'), ST_GeomFr ``` Output: - ``` 0.5 ``` @@ -1806,7 +1780,6 @@ Format: `ST_LineMerge (A: Geometry)` Since: `v1.5.0` Example: - ```sql SELECT ST_LineMerge(ST_GeomFromWKT('MULTILINESTRING ((-29 -27, -30 -29.7, -45 -33), (-45 -33, -46 -32))')) ``` @@ -1920,7 +1893,6 @@ WITH linestring AS ( ``` Result: - ``` +------------------+------------------------+ |st_makevalid(geom)|st_makevalid(geom, true)| @@ -2253,7 +2225,6 @@ Example: SELECT ST_ReducePrecision(ST_GeomFromWKT('Point(0.1234567890123456789 0.1234567890123456789)') , 9) ``` - The new coordinates will only have 9 decimal places. Output: @@ -2370,7 +2341,6 @@ SRID=3021;POLYGON ((1 1, 8 1, 8 8, 1 8, 1 1)) ## ST_SRID - Introduction: Return the spatial reference system identifier (SRID) of the geometry. Format: `ST_SRID (A: Geometry)` @@ -2439,14 +2409,12 @@ Format: `ST_SubDivide(geom: Geometry, maxVertices: Integer)` Since: `v1.5.0` Example: - ```sql SELECT ST_SubDivide(ST_GeomFromText("POLYGON((35 10, 45 45, 15 40, 10 20, 35 10), (20 30, 35 35, 30 20, 20 30))"), 5) ``` Output: - ``` [ POLYGON((37.857142857142854 20, 35 10, 10 20, 37.857142857142854 20)), @@ -2473,7 +2441,6 @@ SELECT ST_SubDivide(ST_GeomFromText("LINESTRING(0 0, 85 85, 100 100, 120 120, 21 ``` Output: - ``` [ LINESTRING(0 0, 5 5) @@ -2799,7 +2766,6 @@ Format: `ST_ZMax(geom: Geometry)` Since: `v1.3.1` Example: - ```sql SELECT ST_ZMax(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` diff --git a/docs/api/snowflake/vector-data/Constructor.md b/docs/api/snowflake/vector-data/Constructor.md index 954a05c2d..ae18496c6 100644 --- a/docs/api/snowflake/vector-data/Constructor.md +++ b/docs/api/snowflake/vector-data/Constructor.md @@ -9,7 +9,6 @@ Format: `ST_GeomFromEWKT (EWkt:string)` SQL example: - ```sql SELECT ST_AsText(ST_GeomFromEWKT('SRID=4269;POINT(40.7128 -74.0060)')) ``` diff --git a/docs/api/snowflake/vector-data/Function.md b/docs/api/snowflake/vector-data/Function.md index 846a44acb..5347dec30 100644 --- a/docs/api/snowflake/vector-data/Function.md +++ b/docs/api/snowflake/vector-data/Function.md @@ -40,7 +40,6 @@ Introduction: Return the 3-dimensional minimum cartesian distance between A and Format: `ST_3DDistance (A:geometry, B:geometry)` SQL example: - ```SQL SELECT ST_3DDistance(polygondf.countyshape, polygondf.countyshape) FROM polygondf @@ -55,7 +54,6 @@ Format: `ST_AddPoint(geom: geometry, point: geometry, position: integer)` Format: `ST_AddPoint(geom: geometry, point: geometry)` SQL example: - ```SQL SELECT ST_AddPoint(ST_GeomFromText('LINESTRING(0 0, 1 1, 1 0)'), ST_GeomFromText('Point(21 52)'), 1) @@ -63,7 +61,6 @@ SELECT ST_AddPoint(ST_GeomFromText('Linestring(0 0, 1 1, 1 0)'), ST_GeomFromText ``` Output: - ``` LINESTRING(0 0, 21 52, 1 1, 1 0) LINESTRING(0 0, 1 1, 1 0, 21 52) @@ -197,7 +194,6 @@ Introduction: Return the area of A Format: `ST_Area (A:geometry)` SQL example: - ```SQL SELECT ST_Area(polygondf.countyshape) FROM polygondf @@ -226,7 +222,6 @@ Introduction: Return the Well-Known Binary representation of a geometry Format: `ST_AsBinary (A:geometry)` SQL example: - ```SQL SELECT ST_AsBinary(polygondf.countyshape) FROM polygondf @@ -243,7 +238,6 @@ If the geometry is lacking SRID a WKB format is produced. Format: `ST_AsEWKB (A:geometry)` SQL example: - ```SQL SELECT ST_AsEWKB(polygondf.countyshape) FROM polygondf @@ -260,7 +254,6 @@ If the geometry is lacking SRID a WKT format is produced. Format: `ST_AsEWKT (A:geometry)` SQL example: - ```SQL SELECT ST_AsEWKT(polygondf.countyshape) FROM polygondf @@ -273,7 +266,6 @@ Introduction: Return the [GeoJSON](https://geojson.org/) string representation o Format: `ST_AsGeoJSON (A:geometry)` SQL example: - ```SQL SELECT ST_AsGeoJSON(polygondf.countyshape) FROM polygondf @@ -286,7 +278,6 @@ Introduction: Return the [GML](https://www.ogc.org/standards/gml) string represe Format: `ST_AsGML (A:geometry)` SQL example: - ```SQL SELECT ST_AsGML(polygondf.countyshape) FROM polygondf @@ -299,7 +290,6 @@ Introduction: Return the [KML](https://www.ogc.org/standards/kml) string represe Format: `ST_AsKML (A:geometry)` SQL example: - ```SQL SELECT ST_AsKML(polygondf.countyshape) FROM polygondf @@ -312,7 +302,6 @@ Introduction: Return the Well-Known Text string representation of a geometry Format: `ST_AsText (A:geometry)` SQL example: - ```SQL SELECT ST_AsText(polygondf.countyshape) FROM polygondf @@ -325,7 +314,6 @@ Introduction: Returns Azimuth for two given points in radians null otherwise. Format: `ST_Azimuth(pointA: Point, pointB: Point)` SQL example: - ```SQL SELECT ST_Azimuth(ST_POINT(0.0, 25.0), ST_POINT(0.0, 0.0)) ``` @@ -339,7 +327,6 @@ Introduction: Returns the closure of the combinatorial boundary of this Geometry Format: `ST_Boundary(geom: geometry)` SQL example: - ```SQL SELECT ST_Boundary(ST_GeomFromText('POLYGON((1 1,0 0, -1 1, 1 1))')) ``` @@ -416,7 +403,6 @@ Introduction: Return the centroid point of A Format: `ST_Centroid (A:geometry)` SQL example: - ```SQL SELECT ST_Centroid(polygondf.countyshape) FROM polygondf @@ -430,7 +416,6 @@ It will throw an exception indicates illegal argument if one of the params is an Format: `ST_ClosestPoint(g1: Geometry, g2: Geometry)` SQL Example: - ```sql SELECT ST_AsText( ST_ClosestPoint(g1, g2)) As ptwkt; ``` @@ -530,7 +515,6 @@ Format: `ST_ConcaveHull (A:geometry, pctConvex:float)` Format: `ST_ConcaveHull (A:geometry, pctConvex:float, allowHoles:Boolean)` SQL example: - ```SQL SELECT ST_ConcaveHull(polygondf.countyshape, pctConvex)` FROM polygondf @@ -543,7 +527,6 @@ Introduction: Return the Convex Hull of polygon A Format: `ST_ConvexHull (A:geometry)` SQL example: - ```SQL SELECT ST_ConvexHull(polygondf.countyshape) FROM polygondf @@ -640,7 +623,6 @@ Introduction: Return the Euclidean distance between A and B Format: `ST_Distance (A:geometry, B:geometry)` SQL example: - ```SQL SELECT ST_Distance(polygondf.countyshape, polygondf.countyshape) FROM polygondf @@ -655,7 +637,6 @@ Geometry must be in EPSG:4326 (WGS84) projection and must be in ==lat/lon== orde Format: `ST_DistanceSphere (A:geometry)` SQL example 1: - ```sql SELECT ST_DistanceSphere(ST_GeomFromWKT('POINT (51.3168 -0.56)'), ST_GeomFromWKT('POINT (55.9533 -3.1883)')) ``` @@ -663,7 +644,6 @@ SELECT ST_DistanceSphere(ST_GeomFromWKT('POINT (51.3168 -0.56)'), ST_GeomFromWKT Output: `543796.9506134904` SQL example 2: - ```sql SELECT ST_DistanceSphere(ST_GeomFromWKT('POINT (51.3168 -0.56)'), ST_GeomFromWKT('POINT (55.9533 -3.1883)'), 6378137.0) ``` @@ -679,7 +659,6 @@ Geometry must be in EPSG:4326 (WGS84) projection and must be in ==lat/lon== orde Format: `ST_DistanceSpheroid (A:geometry)` SQL example: - ```sql SELECT ST_DistanceSpheroid(ST_GeomFromWKT('POINT (51.3168 -0.56)'), ST_GeomFromWKT('POINT (55.9533 -3.1883)')) ``` @@ -693,7 +672,6 @@ Introduction: This function takes a GeometryCollection/Multi Geometry object and Format: `ST_Dump(geom: geometry)` SQL example: - ```SQL SELECT sedona.ST_AsText(geom) FROM table(sedona.ST_Dump(sedona.ST_GeomFromText('MULTIPOINT ((10 40), (40 30), (20 20), (30 10))'))); @@ -715,7 +693,6 @@ Introduction: Returns a MultiPoint geometry which consists of individual points Format: `ST_DumpPoints(geom: geometry)` SQL example: - ```SQL SELECT ST_DumpPoints(ST_GeomFromText('LINESTRING (0 0, 1 1, 1 0)')) ``` @@ -729,7 +706,6 @@ Introduction: Returns last point of given linestring. Format: `ST_EndPoint(geom: geometry)` SQL example: - ```SQL SELECT ST_EndPoint(ST_GeomFromText('LINESTRING(100 150,50 60, 70 80, 160 170)')) ``` @@ -756,7 +732,6 @@ Introduction: Returns a line string representing the exterior ring of the POLYGO Format: `ST_ExteriorRing(geom: geometry)` SQL example: - ```SQL SELECT ST_ExteriorRing(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` @@ -770,7 +745,6 @@ Introduction: Returns a version of the given geometry with X and Y axis flipped. Format: `ST_FlipCoordinates(A:geometry)` SQL example: - ```SQL SELECT ST_FlipCoordinates(df.geometry) FROM df @@ -910,13 +884,11 @@ Format: `ST_GeometricMedian(geom: geometry)` Default parameters: `tolerance: 1e-6, maxIter: 1000, failIfNotConverged: false` Example: - ```sql SELECT ST_GeometricMedian(ST_GeomFromWKT('MULTIPOINT((0 0), (1 1), (2 2), (200 200))')) ``` Output: - ``` POINT (1.9761550281255005 1.9761550281255005) ``` @@ -928,7 +900,6 @@ Introduction: Return the 0-based Nth geometry if the geometry is a GEOMETRYCOLLE Format: `ST_GeometryN(geom: geometry, n: Int)` SQL example: - ```SQL SELECT ST_GeometryN(ST_GeomFromText('MULTIPOINT((1 2), (3 4), (5 6), (8 9))'), 1) ``` @@ -942,7 +913,6 @@ Introduction: Returns the type of the geometry as a string. EG: 'ST_Linestring', Format: `ST_GeometryType (A:geometry)` SQL example: - ```SQL SELECT ST_GeometryType(polygondf.countyshape) FROM polygondf @@ -997,7 +967,6 @@ Introduction: Returns the Nth interior linestring ring of the polygon geometry. Format: `ST_InteriorRingN(geom: geometry, n: Int)` SQL example: - ```SQL SELECT ST_InteriorRingN(ST_GeomFromText('POLYGON((0 0, 0 5, 5 5, 5 0, 0 0), (1 1, 2 1, 2 2, 1 2, 1 1), (1 3, 2 3, 2 4, 1 4, 1 3), (3 3, 4 3, 4 4, 3 4, 3 3))'), 0) ``` @@ -1024,7 +993,6 @@ Introduction: RETURNS true if the LINESTRING start and end point are the same. Format: `ST_IsClosed(geom: geometry)` SQL example: - ```SQL SELECT ST_IsClosed(ST_GeomFromText('LINESTRING(0 0, 1 1, 1 0)')) ``` @@ -1085,7 +1053,6 @@ Introduction: RETURN true if LINESTRING is ST_IsClosed and ST_IsSimple. Format: `ST_IsRing(geom: geometry)` SQL example: - ```SQL SELECT ST_IsRing(ST_GeomFromText('LINESTRING(0 0, 0 1, 1 1, 1 0, 0 0)')) ``` @@ -1113,11 +1080,9 @@ Introduction: Test if a geometry is well-formed. The function can be invoked wit - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValid (A: Geometry) ``` - ``` ST_IsValid (A: Geometry, flag: Integer) ``` @@ -1142,11 +1107,9 @@ Introduction: Returns text stating if the geometry is valid. If not, it provides - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValidReason (A: Geometry) ``` - ``` ST_IsValidReason (A: Geometry, flag: Integer) ``` @@ -1190,7 +1153,6 @@ Introduction: Return the perimeter of A Format: ST_Length (A:geometry) SQL example: - ```SQL SELECT ST_Length(polygondf.countyshape) FROM polygondf @@ -1205,7 +1167,6 @@ Geometry must be in EPSG:4326 (WGS84) projection and must be in ==lat/lon== orde Format: `ST_LengthSpheroid (A:geometry)` SQL example: - ```sql SELECT ST_LengthSpheroid(ST_GeomFromWKT('Polygon ((0 0, 0 90, 0 0))')) ``` @@ -1243,13 +1204,11 @@ Introduction: Returns a point interpolated along a line. First argument must be Format: `ST_LineInterpolatePoint (geom: geometry, fraction: Double)` SQL example: - ```SQL SELECT ST_LineInterpolatePoint(ST_GeomFromWKT('LINESTRING(25 50, 100 125, 150 190)'), 0.2) as Interpolated ``` Output: - ``` +-----------------------------------------+ |Interpolated | @@ -1271,7 +1230,6 @@ SELECT ST_LineLocatePoint(ST_GeomFromWKT('LINESTRING(0 0, 1 1, 2 2)'), ST_GeomFr ``` Output: - ``` 0.5 ``` @@ -1297,13 +1255,11 @@ Introduction: Return a linestring being a substring of the input one starting an Format: `ST_LineSubstring (geom: geometry, startfraction: Double, endfraction: Double)` SQL example: - ```SQL SELECT ST_LineSubstring(ST_GeomFromWKT('LINESTRING(25 50, 100 125, 150 190)'), 0.333, 0.666) as Substring ``` Output: - ``` +------------------------------------------------------------------------------------------------+ |Substring | @@ -1355,7 +1311,6 @@ Format: `ST_MakePolygon(geom: geometry, holes: <geometry>)` Example: Query: - ```SQL SELECT ST_MakePolygon( @@ -1395,7 +1350,6 @@ WITH linestring AS ( ``` Result: - ``` +------------------+------------------------+ |st_makevalid(geom)|st_makevalid(geom, true)| @@ -1417,7 +1371,6 @@ Introduction: Returns the smallest circle polygon that contains a geometry. Format: `ST_MinimumBoundingCircle(geom: geometry, [Optional] quadrantSegments:int)` SQL example: - ```SQL SELECT ST_MinimumBoundingCircle(ST_GeomFromText('POLYGON((1 1,0 0, -1 1, 1 1))')) ``` @@ -1429,7 +1382,6 @@ Introduction: Returns two columns containing the center point and radius of the Format: `ST_MinimumBoundingRadius(geom: geometry)` SQL example: - ```SQL SELECT sedona.ST_AsText(center), radius FROM table(sedona.ST_MinimumBoundingRadius(sedona.ST_GeomFromText('POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))'))) @@ -1571,7 +1523,6 @@ Introduction: RETURNS number of interior rings of polygon geometries. Format: `ST_NumInteriorRings(geom: geometry)` SQL example: - ```SQL SELECT ST_NumInteriorRings(ST_GeomFromText('POLYGON ((0 0, 0 5, 5 5, 5 0, 0 0), (1 1, 2 1, 2 2, 1 2, 1 1))')) ``` @@ -1600,7 +1551,6 @@ IllegalArgumentException: Unsupported geometry type: MultiPoint, only LineString ``` SQL Example: - ```sql SELECT ST_NumPoints(ST_GeomFromText('LINESTRING(0 1, 1 0, 2 0)')) ``` @@ -1614,7 +1564,6 @@ Introduction: Return the Nth point in a single linestring or circular linestring Format: `ST_PointN(geom: geometry, n: integer)` SQL example: - ```SQL SELECT ST_PointN(ST_GeomFromText('LINESTRING(0 0, 1 2, 2 4, 3 6)'), 2) AS geom ``` @@ -1690,7 +1639,6 @@ SQL Example: SELECT ST_ReducePrecision(ST_GeomFromWKT('Point(0.1234567890123456789 0.1234567890123456789)') , 9) ``` - The new coordinates will only have 9 decimal places. Output: @@ -1708,7 +1656,6 @@ Format: `ST_RemovePoint(geom: geometry, position: integer)` Format: `ST_RemovePoint(geom: geometry)` SQL example: - ```SQL SELECT ST_RemovePoint(ST_GeomFromText('LINESTRING(0 0, 1 1, 1 0)'), 1) ``` @@ -1748,13 +1695,11 @@ the cells will be smaller, the coverage will be more accurate, but the result si Format: `ST_S2CellIDs(geom: geometry, level: Int)` SQL example: - ```SQL SELECT ST_S2CellIDs(ST_GeomFromText('LINESTRING(1 3 4, 5 6 7)'), 6) ``` Output: - ``` +------------------------------------------------------------------------------------------------------------------------------+ |st_s2cellids(st_geomfromtext(LINESTRING(1 3 4, 5 6 7), 0), 6) | @@ -1822,7 +1767,6 @@ For example, if a GeometryCollection of only Point geometries is passed as a bla Format: `ST_Split (input: geometry, blade: geometry)` SQL Example: - ```SQL SELECT ST_Split( ST_GeomFromWKT('LINESTRING (0 0, 1.5 1.5, 2 2)'), @@ -1838,7 +1782,6 @@ Introduction: Return the spatial reference system identifier (SRID) of the geome Format: `ST_SRID (A:geometry)` SQL example: - ```SQL SELECT ST_SRID(polygondf.countyshape) FROM polygondf @@ -1851,7 +1794,6 @@ Introduction: Returns first point of given linestring. Format: `ST_StartPoint(geom: geometry)` SQL example: - ```SQL SELECT ST_StartPoint(ST_GeomFromText('LINESTRING(100 150,50 60, 70 80, 160 170)')) ``` @@ -1865,14 +1807,12 @@ Introduction: Returns a multi-geometry divided based of given maximum number of Format: `ST_SubDivide(geom: geometry, maxVertices: int)` SQL example: - ```SQL SELECT Sedona.ST_AsText(Sedona.ST_SubDivide(Sedona.ST_GeomFromText('LINESTRING(0 0, 85 85, 100 100, 120 120, 21 21, 10 10, 5 5)'), 5)); ``` Output: - ``` MULTILINESTRING ((0 0, 5 5), (5 5, 10 10), (10 10, 21 21), (21 21, 60 60), (60 60, 85 85), (85 85, 100 100), (100 100, 120 120)) ``` @@ -1888,7 +1828,6 @@ FROM table(SEDONA.ST_SubDivideExplode(geom: geometry, maxVertices: int))` Example: Query: - ```SQL SELECT Sedona.ST_AsText(GEOM) FROM table(Sedona.ST_SubDivideExplode(Sedona.ST_GeomFromText('LINESTRING(0 0, 85 85, 100 100, 120 120, 21 21, 10 10, 5 5)'), 5)); @@ -1944,14 +1883,12 @@ For SourceCRS and TargetCRS, WKT format is also available. Format: `ST_Transform (A:geometry, SourceCRS:string, TargetCRS:string ,[Optional] DisableError)` SQL example (simple): - ```SQL SELECT ST_Transform(polygondf.countyshape, 'epsg:4326','epsg:3857') FROM polygondf ``` SQL example (with optional parameters): - ```SQL SELECT ST_Transform(polygondf.countyshape, 'epsg:4326','epsg:3857', false) FROM polygondf @@ -2030,7 +1967,6 @@ Introduction: Returns X Coordinate of given Point null otherwise. Format: `ST_X(pointA: Point)` SQL example: - ```SQL SELECT ST_X(ST_POINT(0.0 25.0)) ``` @@ -2078,7 +2014,6 @@ Introduction: Returns Y Coordinate of given Point, null otherwise. Format: `ST_Y(pointA: Point)` SQL example: - ```SQL SELECT ST_Y(ST_POINT(0.0 25.0)) ``` @@ -2092,7 +2027,6 @@ Introduction: Return the minimum Y coordinate of A Format: `ST_YMax (A:geometry)` SQL example: - ```SQL SELECT ST_YMax(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` @@ -2106,7 +2040,6 @@ Introduction: Return the minimum Y coordinate of A Format: `ST_Y_Min (A:geometry)` SQL example: - ```SQL SELECT ST_YMin(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` @@ -2120,7 +2053,6 @@ Introduction: Returns Z Coordinate of given Point, null otherwise. Format: `ST_Z(pointA: Point)` SQL example: - ```SQL SELECT ST_Z(ST_POINT(0.0 25.0 11.0)) ``` @@ -2134,7 +2066,6 @@ Introduction: Returns Z maxima of the given geometry or null if there is no Z co Format: `ST_ZMax(geom: geometry)` SQL example: - ```SQL SELECT ST_ZMax(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` @@ -2148,7 +2079,6 @@ Introduction: Returns Z minima of the given geometry or null if there is no Z co Format: `ST_ZMin(geom: geometry)` SQL example: - ```SQL SELECT ST_ZMin(ST_GeomFromText('LINESTRING(1 3 4, 5 6 7)')) ``` diff --git a/docs/api/sql/Constructor.md b/docs/api/sql/Constructor.md index 6498e5eb4..cd5b667f4 100644 --- a/docs/api/sql/Constructor.md +++ b/docs/api/sql/Constructor.md @@ -54,7 +54,6 @@ Format: Since: `v1.5.0` SQL example: - ```sql SELECT ST_AsText(ST_GeomFromEWKT('SRID=4269;POINT(40.7128 -74.0060)')) ``` diff --git a/docs/api/sql/DataFrameAPI.md b/docs/api/sql/DataFrameAPI.md index aaa5b9044..a9b5e2b89 100644 --- a/docs/api/sql/DataFrameAPI.md +++ b/docs/api/sql/DataFrameAPI.md @@ -50,7 +50,6 @@ A short example of using this API (uses the `array_min` and `array_max` Spark fu ``` The above code will generate the following dataframe: - ``` +-----------+ |point | @@ -69,7 +68,6 @@ df = df.select(ST_Point(1.0, 3.0).alias("point")) ``` This will generate a dataframe with a constant point in a column: - ``` +-----------+ |point | diff --git a/docs/api/sql/Function.md b/docs/api/sql/Function.md index 82bc3be8b..b64412250 100644 --- a/docs/api/sql/Function.md +++ b/docs/api/sql/Function.md @@ -212,13 +212,11 @@ Format: `ST_Area (A: Geometry)` Since: `v1.0.0` SQL Example - ```sql SELECT ST_Area(ST_GeomFromText("POLYGON(0 0, 0 10, 10 10, 0 10, 0 0)")) ``` Output: - ``` 10 ``` @@ -507,7 +505,6 @@ Format: `ST_BoundingDiagonal(geom: Geometry)` Since: `v1.5.0` SQL Example - ```sql SELECT ST_BoundingDiagonal(ST_GeomFromWKT(geom)) ``` @@ -631,7 +628,6 @@ Format: `ST_ClosestPoint(g1: Geometry, g2: Geometry)` Since: `v1.5.0` SQL Example - ```sql SELECT ST_AsText( ST_ClosestPoint(g1, g2)) As ptwkt; ``` @@ -970,7 +966,6 @@ Format: `ST_Dump(geom: Geometry)` Since: `v1.0.0` SQL Example - ```sql SELECT ST_Dump(ST_GeomFromText('MULTIPOINT ((10 40), (40 30), (20 20), (30 10))')) ``` @@ -1225,13 +1220,11 @@ Default parameters: `tolerance: 1e-6, maxIter: 1000, failIfNotConverged: false` Since: `v1.4.1` SQL Example - ```sql SELECT ST_GeometricMedian(ST_GeomFromWKT('MULTIPOINT((0 0), (1 1), (2 2), (200 200))')) ``` Output: - ``` POINT (1.9761550281255005 1.9761550281255005) ``` @@ -1289,13 +1282,11 @@ Format: `ST_H3CellDistance(cell1: Long, cell2: Long)` Since: `v1.5.0` SQL Example - ```sql select ST_H3CellDistance(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[0], ST_H3CellIDs(ST_GeomFromWKT('POINT(1.23 1.59)'), 8, true)[0]) ``` Output: - ``` +-----------------------------------------------------------------------------------------------------------------------------------------+ |st_h3celldistance(st_h3cellids(st_geomfromwkt(POINT(1 2), 0), 8, true)[0], st_h3cellids(st_geomfromwkt(POINT(1.23 1.59), 0), 8, true)[0])| @@ -1338,13 +1329,11 @@ Format: `ST_H3CellIDs(geom: geometry, level: Int, fullCover: Boolean)` Since: `v1.5.0` SQL Example - ```sql SELECT ST_H3CellIDs(ST_GeomFromText('LINESTRING(1 3 4, 5 6 7)'), 6, true) ``` Output: - ``` +-------------------------------------------------------------+ |st_h3cellids(st_geomfromtext(LINESTRING(1 3 4, 5 6 7), 0), 6)| @@ -1367,13 +1356,11 @@ Format: `ST_H3KRing(cell: Long, k: Int, exactRing: Boolean)` Since: `v1.5.0` SQL Example - ```sql SELECT ST_H3KRing(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[0], 1, true) cells union select ST_H3KRing(ST_H3CellIDs(ST_GeomFromWKT('POINT(1 2)'), 8, true)[0], 1, false) cells ``` Output: - ``` +--------------------------------------------------------------------------------------------------------------------------------------------+ |cells | @@ -1617,11 +1604,9 @@ Introduction: Test if a geometry is well-formed. The function can be invoked wit - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValid (A: Geometry) ``` - ``` ST_IsValid (A: Geometry, flag: Integer) ``` @@ -1648,11 +1633,9 @@ Introduction: Returns text stating if the geometry is valid. If not, it provides - 1: "ESRI flag", Accepts certain self-touching rings as valid, which are considered invalid under OGC standards. Formats: - ``` ST_IsValidReason (A: Geometry) ``` - ``` ST_IsValidReason (A: Geometry, flag: Integer) ``` @@ -1771,7 +1754,6 @@ SELECT ST_LineInterpolatePoint(ST_GeomFromWKT('LINESTRING(25 50, 100 125, 150 19 ``` Output: - ``` POINT (51.5974135047432 76.5974135047432) ``` @@ -1791,7 +1773,6 @@ SELECT ST_LineLocatePoint(ST_GeomFromWKT('LINESTRING(0 0, 1 1, 2 2)'), ST_GeomFr ``` Output: - ``` 0.5 ``` @@ -1924,7 +1905,6 @@ WITH linestring AS ( ``` Result: - ``` +------------------+------------------------+ |st_makevalid(geom)|st_makevalid(geom, true)| @@ -2181,7 +2161,6 @@ Format: `ST_PointN(geom: Geometry, n: Integer)` Since: `v1.2.1` SQL Example - ```sql SELECT ST_PointN(ST_GeomFromText("LINESTRING(0 0, 1 2, 2 4, 3 6)"), 2) ``` @@ -2259,7 +2238,6 @@ SQL Example SELECT ST_ReducePrecision(ST_GeomFromWKT('Point(0.1234567890123456789 0.1234567890123456789)') , 9) ``` - The new coordinates will only have 9 decimal places. Output: @@ -2329,7 +2307,6 @@ SELECT ST_S2CellIDs(ST_GeomFromText('LINESTRING(1 3 4, 5 6 7)'), 6) ``` Output: - ``` [1159395429071192064, 1159958379024613376, 1160521328978034688, 1161084278931456000, 1170091478186196992, 1170654428139618304] ``` @@ -2472,14 +2449,12 @@ Format: `ST_SubDivide(geom: Geometry, maxVertices: Integer)` Since: `v1.1.0` SQL Example - ```sql SELECT ST_SubDivide(ST_GeomFromText("POLYGON((35 10, 45 45, 15 40, 10 20, 35 10), (20 30, 35 35, 30 20, 20 30))"), 5) ``` Output: - ``` [ POLYGON((37.857142857142854 20, 35 10, 10 20, 37.857142857142854 20)), @@ -2506,7 +2481,6 @@ SELECT ST_SubDivide(ST_GeomFromText("LINESTRING(0 0, 85 85, 100 100, 120 120, 21 ``` Output: - ``` [ LINESTRING(0 0, 5 5) @@ -2530,7 +2504,6 @@ Since: `v1.1.0` SQL Example Query: - ```sql SELECT ST_SubDivideExplode(ST_GeomFromText("LINESTRING(0 0, 85 85, 100 100, 120 120, 21 21, 10 10, 5 5)"), 5) ``` @@ -2554,7 +2527,6 @@ Result: Using Lateral View Table: - ``` +-------------------------------------------------------------+ |geometry | @@ -2564,13 +2536,11 @@ Table: ``` Query - ```sql select geom from geometries LATERAL VIEW ST_SubdivideExplode(geometry, 5) AS geom ``` Result: - ``` +-----------------------------+ |geom | @@ -2865,7 +2835,6 @@ Format: `ST_YMax (A: Geometry)` Since: `v1.2.1` SQL Example - ```sql SELECT ST_YMax(ST_GeomFromText('POLYGON((0 0 1, 1 1 1, 1 2 1, 1 1 1, 0 0 1))')) ``` diff --git a/docs/api/sql/Optimizer.md b/docs/api/sql/Optimizer.md index 1adc2b2fd..0e817cbc6 100644 --- a/docs/api/sql/Optimizer.md +++ b/docs/api/sql/Optimizer.md @@ -52,7 +52,6 @@ Introduction: Find geometries from A and geometries from B such that the distanc Spark SQL Example for planar Euclidean distance: *Only consider ==fully within a certain distance==* - ```sql SELECT * FROM pointdf1, pointdf2 @@ -72,7 +71,6 @@ WHERE ST_FrechetDistance(pointDf.pointshape, polygonDf.polygonshape) < 2 ``` *Consider ==intersects within a certain distance==* - ```sql SELECT * FROM pointdf1, pointdf2 @@ -92,7 +90,6 @@ WHERE ST_FrechetDistance(pointDf.pointshape, polygonDf.polygonshape) <= 2 ``` Spark SQL Physical plan: - ``` == Physical Plan == DistanceJoin pointshape1#12: geometry, pointshape2#33: geometry, 2.0, true @@ -108,7 +105,6 @@ DistanceJoin pointshape1#12: geometry, pointshape2#33: geometry, 2.0, true Spark SQL Example for meter-based geodesic distance `ST_DistanceSpheroid` (works for `ST_DistanceSphere` too): *Less than a certain distance==* - ```sql SELECT * FROM pointdf1, pointdf2 @@ -116,7 +112,6 @@ WHERE ST_DistanceSpheroid(pointdf1.pointshape1,pointdf2.pointshape2) < 2 ``` *Less than or equal to a certain distance==* - ```sql SELECT * FROM pointdf1, pointdf2 @@ -146,7 +141,6 @@ pointDf.alias("pointDf").join(broadcast(polygonDf).alias("polygonDf"), expr("ST_ ``` Spark SQL Physical plan: - ``` == Physical Plan == BroadcastIndexJoin pointshape#52: geometry, BuildRight, BuildRight, false ST_Contains(polygonshape#30, pointshape#52) @@ -164,7 +158,6 @@ pointDf1.alias("pointDf1").join(broadcast(pointDf2).alias("pointDf2"), expr("ST_ ``` Spark SQL Physical plan: - ``` == Physical Plan == BroadcastIndexJoin pointshape#52: geometry, BuildRight, BuildLeft, true, 2.0 ST_Distance(pointshape#52, pointshape#415) <= 2.0 diff --git a/docs/api/sql/Overview.md b/docs/api/sql/Overview.md index 52bf11345..84093470e 100644 --- a/docs/api/sql/Overview.md +++ b/docs/api/sql/Overview.md @@ -3,13 +3,11 @@ ## Function list SedonaSQL supports SQL/MM Part3 Spatial SQL Standard. It includes four kinds of SQL operators as follows. All these operators can be directly called through: - ```scala var myDataFrame = sedona.sql("YOUR_SQL") ``` Alternatively, `expr` and `selectExpr` can be used: - ```scala myDataFrame.withColumn("geometry", expr("ST_*")).selectExpr("ST_*") ``` @@ -37,16 +35,13 @@ The detailed explanation is here [Write a SQL/DataFrame application](../../tutor 1. Add Sedona-core and Sedona-SQL into your project pom.xml or build.sbt 2. Create your Sedona config if you want to customize your SparkSession. - ```scala import org.apache.sedona.spark.SedonaContext val config = SedonaContext.builder(). master("local[*]").appName("SedonaSQL") .getOrCreate() ``` - 3. Add the following line after your Sedona context declaration: - ```scala import org.apache.sedona.spark.SedonaContext val sedona = SedonaContext.create(config) diff --git a/docs/api/sql/Parameter.md b/docs/api/sql/Parameter.md index b84b7cdee..4ce7beefd 100644 --- a/docs/api/sql/Parameter.md +++ b/docs/api/sql/Parameter.md @@ -3,7 +3,6 @@ SedonaSQL supports many parameters. To change their values, 1. Set it through SparkConf: - ```scala sparkSession = SparkSession.builder(). config("spark.serializer","org.apache.spark.serializer.KryoSerializer"). @@ -11,16 +10,12 @@ sparkSession = SparkSession.builder(). config("sedona.global.index","true") master("local[*]").appName("mySedonaSQLdemo").getOrCreate() ``` - 2. Check your current SedonaSQL configuration: - ```scala val sedonaConf = new SedonaConf(sparkSession.conf) println(sedonaConf) ``` - 3. Sedona parameters can be changed at runtime: - ```scala sparkSession.conf.set("sedona.global.index","false") ``` diff --git a/docs/api/sql/Raster-loader.md b/docs/api/sql/Raster-loader.md index d8b966aef..2032fd35c 100644 --- a/docs/api/sql/Raster-loader.md +++ b/docs/api/sql/Raster-loader.md @@ -159,7 +159,6 @@ SELECT RS_MakeEmptyRaster(2, 'F', 10, 10, 0.0, 0.0, 1.0, -1.0, 0.0, 0.0, 4326) - ``` Output: - ``` +------------------------------------------------------------------+ |rs_makeemptyraster(2, 10, 10, 0.0, 0.0, 1.0, -1.0, 0.0, 0.0, 4326)| diff --git a/docs/api/sql/Raster-operators.md b/docs/api/sql/Raster-operators.md index cd08a45bc..93c7066bd 100644 --- a/docs/api/sql/Raster-operators.md +++ b/docs/api/sql/Raster-operators.md @@ -32,13 +32,11 @@ Format: `RS_PixelAsCentroids(raster: Raster, band: Integer)` Since: `v1.5.1` SQL Example - ```sql SELECT ST_AsText(RS_PixelAsCentroids(raster, 1)) from rasters ``` Output: - ``` [[POINT (-13065222 4021263.75),148.0,0,0], [POINT (-13065151 4021263.75),123.0,0,1], [POINT (-13065077 4021263.75),99.0,1,0], [POINT (-13065007 4021261.75),140.0,1,1]] ``` @@ -91,7 +89,6 @@ SELECT ST_AsText(RS_PixelAsPoint(raster, 2, 1)) from rasters ``` Output: - ``` POINT (123.19, -12) ``` @@ -103,7 +100,6 @@ SELECT ST_AsText(RS_PixelAsPoint(raster, 6, 2)) from rasters ``` Output: - ``` IndexOutOfBoundsException: Specified pixel coordinates (6, 2) do not lie in the raster ``` @@ -117,13 +113,11 @@ Format: `RS_PixelAsPoints(raster: Raster, band: Integer)` Since: `v1.5.1` SQL Example - ```sql SELECT ST_AsText(RS_PixelAsPoints(raster, 1)) from rasters ``` Output: - ``` [[POINT (-13065223 4021262.75),148.0,0,0], [POINT (-13065150 4021262.75),123.0,0,1], [POINT (-13065078 4021262.75),99.0,1,0], [POINT (-13065006 4021262.75),140.0,1,1]] ``` @@ -189,13 +183,11 @@ Format: `RS_PixelAsPolygons(raster: Raster, band: Integer)` Since: `v1.5.1` SQL Example - ```sql SELECT ST_AsText(RS_PixelAsPolygons(raster, 1)) from rasters ``` Output: - ``` [[POLYGON ((123.19000244140625 -12, 127.19000244140625 -12, 127.19000244140625 -16, 123.19000244140625 -16, 123.19000244140625 -12)),0.0,1,1], [POLYGON ((127.19000244140625 -12, 131.19000244140625 -12, 131.19000244140625 -16, 127.19000244140625 -16, 127.19000244140625 -12)),0.0,2,1], @@ -302,7 +294,6 @@ inputDf.selectExpr("ST_AsText(RS_MinConvexHull(RS_AddBandFromArray(" + ``` Output: - ```sql +----------------------------------------+ |minConvexHullAll | @@ -321,7 +312,6 @@ inputDf.selectExpr("ST_AsText(RS_MinConvexHull(RS_AddBandFromArray(" + ``` Output: - ```sql +----------------------------------------+ |minConvexHull1 | @@ -635,7 +625,6 @@ SELECT RS_ScaleY(raster) FROM rasters ``` Output: - ``` -2 ``` @@ -735,7 +724,6 @@ SELECT RS_Width(raster) FROM rasters ``` Output: - ``` 517 ``` @@ -1259,7 +1247,6 @@ SELECT RS_Contains(RS_MakeEmptyRaster(1, 20, 20, 2, 22, 1), ST_GeomFromWKT('POLY ``` Output: - ``` +---------+---------+ |rast_geom|rast_rast| @@ -1329,7 +1316,6 @@ SELECT RS_Within(RS_MakeEmptyRaster(1, 20, 20, 2, 22, 1), ST_GeomFromWKT('POLYGO ``` Output: - ``` +---------+---------+ |rast_geom|rast_rast| @@ -1755,7 +1741,6 @@ Output: ### RS_SetValue - Introduction: Returns a raster by replacing the value of pixel specified by `colX` and `rowY`. Format: @@ -2187,7 +2172,6 @@ SELECT RS_AddBandFromArray(raster, RS_MultiplyFactor(RS_BandAsArray(RS_FromGeoTi ``` Output: - ``` +--------------------+ | raster| @@ -2230,7 +2214,6 @@ SELECT RS_MapAlgebra(rast, 'D', 'out = (rast[3] - rast[0]) / (rast[3] + rast[0]) ``` Output: - ``` +--------------------+ | raster| diff --git a/docs/api/sql/Raster-visualizer.md b/docs/api/sql/Raster-visualizer.md index 7dc20a94f..68f4d4efb 100644 --- a/docs/api/sql/Raster-visualizer.md +++ b/docs/api/sql/Raster-visualizer.md @@ -135,7 +135,6 @@ print(inputDf.selectExpr("RS_AsMatrix(RS_AddBandFromArray(RS_MakeEmptyRaster(1, ``` Output: - ```sql | 1 3 4 0| | 2 9 10 11| diff --git a/docs/api/sql/Visualization_SedonaKepler.md b/docs/api/sql/Visualization_SedonaKepler.md index ef97d7cea..a329954b8 100644 --- a/docs/api/sql/Visualization_SedonaKepler.md +++ b/docs/api/sql/Visualization_SedonaKepler.md @@ -1,13 +1,11 @@ SedonaKepler offers a number of APIs which aid in quick and interactive visualization of a geospatial data in a Jupyter notebook/lab environment. Inorder to start using SedonaKepler, simply import Sedona using: - ```python from sedona.spark import * ``` Alternatively it can also be imported using: - ```python from sedona.maps.SedonaKepler import SedonaKepler ``` diff --git a/docs/api/sql/Visualization_SedonaPyDeck.md b/docs/api/sql/Visualization_SedonaPyDeck.md index d8b9d07eb..26e555cce 100644 --- a/docs/api/sql/Visualization_SedonaPyDeck.md +++ b/docs/api/sql/Visualization_SedonaPyDeck.md @@ -1,13 +1,11 @@ SedonaPyDeck offers a number of APIs which aid in quick and interactive visualization of a geospatial data in a Jupyter notebook/lab environment. Inorder to start using SedonaPyDeck, simply import Sedona using: - ```python from sedona.spark import * ``` Alternatively it can also be imported using: - ```python from sedona.maps.SedonaPyDeck import SedonaPyDeck ``` @@ -41,7 +39,6 @@ def create_choropleth_map(df, fill_color=None, plot_col=None, initial_view_state The parameter `fill_color` can be given a list of RGB/RGBA values, or a string that contains RGB/RGBA values based on a column. For example, all these are valid values of fill_color: - ```python fill_color=[255, 12, 250] fill_color=[0, 12, 250, 255] diff --git a/docs/api/viz/sql.md b/docs/api/viz/sql.md index 015019761..67582cec0 100644 --- a/docs/api/viz/sql.md +++ b/docs/api/viz/sql.md @@ -4,16 +4,13 @@ The detailed explanation is here: [Visualize Spatial DataFrame/RDD](../../tutori 1. Add Sedona-core, Sedona-SQL, Sedona-Viz into your project pom.xml or build.sbt 2. Declare your Spark Session - ```scala sparkSession = SparkSession.builder(). config("spark.serializer","org.apache.spark.serializer.KryoSerializer"). config("spark.kryo.registrator", "org.apache.sedona.viz.core.Serde.SedonaVizKryoRegistrator"). master("local[*]").appName("mySedonaVizDemo").getOrCreate() ``` - 3. Add the following lines after your SparkSession declaration: - ```scala SedonaSQLRegistrator.registerAll(sparkSession) SedonaVizRegistrator.registerAll(sparkSession) @@ -41,7 +38,6 @@ Since: `v1.0.0` This function will normalize the weight according to the max weight among all pixels. Different pixel obtains different color. SQL Example - ```sql SELECT pixels.px, ST_Colorize(pixels.weight, 999) AS color FROM pixels @@ -59,7 +55,6 @@ FROM pixels ``` Here are some example color names can be entered: - ``` "firebrick" "#aa38e0" diff --git a/docs/community/publish.md b/docs/community/publish.md index e4a2f980d..1e574e04a 100644 --- a/docs/community/publish.md +++ b/docs/community/publish.md @@ -8,12 +8,10 @@ This page is for Sedona PMC to publish Sedona releases. ## 0. Prepare an empty script file 1. In your local Sedona Git repo under master branch, run - ```bash echo '#!/bin/bash' > create-release.sh chmod 777 create-release.sh ``` - 2. Use your favourite GUI text editor to open `create-release.sh`. 3. Then keep copying the scripts on this web page to replace all content in this script file. 4. Do NOT directly copy/paste the scripts to your terminal because a bug in `clipboard.js` will create link breaks in such case. @@ -22,7 +20,6 @@ chmod 777 create-release.sh ## 1. Check ASF copyright in all file headers 1. Run the following script: - ```bash #!/bin/bash wget -q https://archive.apache.org/dist/creadur/apache-rat-0.15/apache-rat-0.15-bin.tar.gz @@ -30,10 +27,8 @@ tar -xvf apache-rat-0.15-bin.tar.gz git clone --shared --branch master https://github.com/apache/sedona.git sedona-src java -jar apache-rat-0.15/apache-rat-0.15.jar -d sedona-src > report.txt ``` - 2. Read the generated report.txt file and make sure all source code files have ASF header. 3. Delete the generated report and cloned files - ```bash #!/bin/bash rm -rf apache-rat-0.15 diff --git a/docs/community/release-manager.md b/docs/community/release-manager.md index 9b90d1af4..c02c712ca 100644 --- a/docs/community/release-manager.md +++ b/docs/community/release-manager.md @@ -13,19 +13,15 @@ If your Maven (`mvn --version`) points to other JDK versions, you must change it 1. Find all Java installed on your machine: `/usr/libexec/java_home -V`. You should see multiple JDK versions including JDK 8. 2. Run `whereis mvn` to get the installation location of your Maven. The result is a symlink to the actual location. 3. Open it in the terminal (with `sudo` if needed). It will be like this - ``` #!/bin/bash JAVA_HOME="${JAVA_HOME:-$(/usr/libexec/java_home)}" exec "/usr/local/Cellar/maven/3.6.3/libexec/bin/mvn" "$@" ``` - 4. Change `JAVA_HOME:-$(/usr/libexec/java_home)}` to `JAVA_HOME:-$(/usr/libexec/java_home -v 1.8)}`. The resulting content will be like this: - ``` #!/bin/bash JAVA_HOME="${JAVA_HOME:-$(/usr/libexec/java_home -v 1.8)}" exec "/usr/local/Cellar/maven/3.6.3/libexec/bin/mvn" "$@" ``` - 5. Run `mvn --version` again. It should now point to JDK 8. ## 1. Obtain Write Access to Sedona GitHub repo @@ -63,23 +59,18 @@ Use SVN to append your armored PGP public key to the `KEYS` files * https://dist.apache.org/repos/dist/release/sedona/KEYS 1. Check out both KEYS files - ```bash svn checkout https://dist.apache.org/repos/dist/dev/sedona/ sedona-dev --depth files svn checkout https://dist.apache.org/repos/dist/release/sedona/ sedona-release --depth files ``` - 2. Use your favorite text editor to open `sedona-dev/KEYS` and `sedona-release/KEYS`. 3. Paste your armored key to the end of both files. Note: There must be an empty line between `-----BEGIN PGP PUBLIC KEY BLOCK-----` and the actual key. 4. Commit both KEYS. SVN might ask you to enter your ASF ID and password. Make sure you do it so SVN can always store your ID and password locally. - ```bash svn commit -m "Update KEYS" sedona-dev/KEYS svn commit -m "Update KEYS" sedona-release/KEYS ``` - 5. Then remove both svn folders - ```bash rm -rf sedona-dev rm -rf sedona-release diff --git a/docs/community/snapshot.md b/docs/community/snapshot.md index 88be2bc85..8dbed0ce7 100644 --- a/docs/community/snapshot.md +++ b/docs/community/snapshot.md @@ -12,12 +12,10 @@ The detailed requirement is on [ASF Infra website](https://infra.apache.org/publ ## 0. Prepare an empty script file 1. In your local Sedona Git repo under master branch, run - ```bash echo '#!/bin/bash' > create-release.sh chmod 777 create-release.sh ``` - 2. Use your favourite GUI text editor to open `create-release.sh`. 3. Then keep copying the scripts on this web page to replace all content in this text file. 4. Do NOT directly copy/paste the scripts to your terminal because a bug in `clipboard.js` will create link breaks in such case. diff --git a/docs/setup/databricks.md b/docs/setup/databricks.md index 1e26805f6..9641e4c47 100644 --- a/docs/setup/databricks.md +++ b/docs/setup/databricks.md @@ -25,14 +25,12 @@ This method cannot achieve the best performance of Sedona and does not work for ### Install libraries 1) From the Libraries tab install from Maven Coordinates - ``` org.apache.sedona:sedona-spark-shaded-3.0_2.12:{{ sedona.current_version }} org.datasyslab:geotools-wrapper:{{ sedona.current_geotools }} ``` 2) For enabling python support, from the Libraries tab install from PyPI - ``` apache-sedona keplergl==0.3.2 @@ -44,14 +42,12 @@ This method cannot achieve the best performance of Sedona and does not work for After you have installed the libraries and started the cluster, you can initialize the Sedona `ST_*` functions and types by running from your code: (scala) - ```scala import org.apache.sedona.sql.utils.SedonaSQLRegistrator SedonaSQLRegistrator.registerAll(spark) ``` (or python) - ```python from sedona.register.geo_registrator import SedonaRegistrator SedonaRegistrator.registerAll(spark) @@ -108,7 +104,6 @@ EOF ### Set up cluster config From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Advanced options` -> `Spark`) activate the Sedona functions and the kryo serializer by adding to the Spark Config - ``` spark.sql.extensions org.apache.sedona.viz.sql.SedonaVizExtensions,org.apache.sedona.sql.SedonaSqlExtensions spark.serializer org.apache.spark.serializer.KryoSerializer @@ -116,13 +111,11 @@ spark.kryo.registrator org.apache.sedona.core.serde.SedonaKryoRegistrator ``` From your cluster configuration (`Cluster` -> `Edit` -> `Configuration` -> `Advanced options` -> `Init Scripts`) add the newly created `Workspace` init script - ``` /Workspace/sedona/sedona-init.sh ``` For enabling python support, from the Libraries tab install from PyPI - ``` apache-sedona=={{ sedona.current_version }} geopandas==0.11.1 diff --git a/docs/setup/docker.md b/docs/setup/docker.md index f5a4f21b5..d510651dd 100644 --- a/docs/setup/docker.md +++ b/docs/setup/docker.md @@ -33,7 +33,6 @@ Format: ```bash docker run -e DRIVER_MEM=<driver_mem> -e EXECUTOR_MEM=<executor_mem> -p 8888:8888 -p 8080:8080 -p 8081:8081 -p 4040:4040 apache/sedona:<sedona_version> ``` - Driver memory and executor memory are optional. If their values are not given, the container will take 4GB RAM for the driver and 4GB RAM for the executor. Example 1: diff --git a/docs/setup/flink/install-scala.md b/docs/setup/flink/install-scala.md index 820db46c8..d99a74670 100644 --- a/docs/setup/flink/install-scala.md +++ b/docs/setup/flink/install-scala.md @@ -8,7 +8,6 @@ To use Sedona in your self-contained Flink project, you just need to add Sedona 2. Read [Sedona Flink guide](../../tutorial/flink/sql.md) and use Sedona Template project to start: [Sedona Template Project](../../tutorial/demo.md) 3. Compile your project using Maven. Make sure you obtain the fat jar which packages all dependencies. 4. Submit your compiled fat jar to Flink cluster. Make sure you are in the root folder of Flink distribution. Then run the following command: - ``` ./bin/flink run /Path/To/YourJar.jar ``` diff --git a/docs/setup/install-scala.md b/docs/setup/install-scala.md index 83bca4b10..1262b41ee 100644 --- a/docs/setup/install-scala.md +++ b/docs/setup/install-scala.md @@ -12,11 +12,9 @@ There are two ways to use a Scala or Java library with Apache Spark. You can use 1. Have your Spark cluster ready. 2. Run Spark shell with `--packages` option. This command will automatically download Sedona jars from Maven Central. - ``` ./bin/spark-shell --packages MavenCoordinates ``` - Please refer to [Sedona Maven Central coordinates](maven-coordinates.md) to select the corresponding Sedona packages for your Spark version. * Local mode: test Sedona without setting up a cluster @@ -37,11 +35,9 @@ Please refer to [Sedona Maven Central coordinates](maven-coordinates.md) to sele * Download the pre-compiled jars from [Sedona Releases](../download.md) * Download / Git clone Sedona source code and compile the code by yourself (see [Compile Sedona](compile.md)) 3. Run Spark shell with `--jars` option. - ``` ./bin/spark-shell --jars /Path/To/SedonaJars.jar ``` - If you are using Spark 3.0 to 3.3, please use jars with filenames containing `3.0`, such as `sedona-spark-shaded-3.0_2.12-{{ sedona.current_version }}`; If you are using Spark 3.4 or higher versions, please use jars with Spark major.minor versions in the filename, such as `sedona-spark-shaded-3.4_2.12-{{ sedona.current_version }}`. * Local mode: test Sedona without setting up a cluster @@ -66,7 +62,6 @@ A self-contained project allows you to create multiple Scala / Java files and wr 2. Use Sedona Template project to start: [Sedona Template Project](../tutorial/demo.md) 3. Compile your project using SBT. Make sure you obtain the fat jar which packages all dependencies. 4. Submit your compiled fat jar to Spark cluster. Make sure you are in the root folder of Spark distribution. Then run the following command: - ``` ./bin/spark-submit --master spark://YOUR-IP:7077 /Path/To/YourJar.jar ``` diff --git a/docs/setup/release-notes.md b/docs/setup/release-notes.md index acaf11075..ed9e5754e 100644 --- a/docs/setup/release-notes.md +++ b/docs/setup/release-notes.md @@ -444,15 +444,12 @@ Sedona 1.4.1 is compiled against Spark 3.3 / Spark 3.4 / Flink 1.12, Java 8. * **Sedona Spark & Flink** Introduced a new entry point called SedonaContext to unify all Sedona entry points in different compute engines and deprecate old Sedona register entry points. Users no longer have to register Sedona kryo serializer and import many tedious Python classes. * **Sedona Spark**: * Scala: - ```scala import org.apache.sedona.spark.SedonaContext val sedona = SedonaContext.create(SedonaContext.builder().master("local[*]").getOrCreate()) sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM") ``` - * Python: - ```python from sedona.spark import * @@ -464,9 +461,7 @@ Sedona 1.4.1 is compiled against Spark 3.3 / Spark 3.4 / Flink 1.12, Java 8. sedona = SedonaContext.create(config) sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM") ``` - * **Sedona Flink**: - ```java import org.apache.sedona.flink.SedonaContext StreamTableEnvironment sedona = SedonaContext.create(env, tableEnv); diff --git a/docs/tutorial/Advanced-Tutorial-Tune-your-Application.md b/docs/tutorial/Advanced-Tutorial-Tune-your-Application.md index d9b89d13e..58e31e43c 100644 --- a/docs/tutorial/Advanced-Tutorial-Tune-your-Application.md +++ b/docs/tutorial/Advanced-Tutorial-Tune-your-Application.md @@ -17,13 +17,10 @@ The third level (i.e., 0.8.1) tells that this version only contains bug fixes, s Sedona provides a number of constructors for each SpatialRDD (PointRDD, PolygonRDD and LineStringRDD). In general, you have two options to start with. 1. Initialize a SpatialRDD from your data source such as HDFS and S3. A typical example is as follows: - ```java public PointRDD(JavaSparkContext sparkContext, String InputLocation, Integer Offset, FileDataSplitter splitter, boolean carryInputData, Integer partitions, StorageLevel newLevel) ``` - 2. Initialize a SpatialRDD from an existing RDD. A typical example is as follows: - ```java public PointRDD(JavaRDD<Point> rawSpatialRDD, StorageLevel newLevel) ``` @@ -35,7 +32,6 @@ However, in some cases, you may know well about your datasets. If so, you can ma ```java public PointRDD(JavaSparkContext sparkContext, String InputLocation, Integer Offset, FileDataSplitter splitter, boolean carryInputData, Integer partitions, Envelope datasetBoundary, Integer approximateTotalCount) { ``` - Manually providing the dataset boundary and approximate total count helps Sedona avoiding several slow "Action"s during initialization. ## Cache the Spatial RDD that is repeatedly used diff --git a/docs/tutorial/flink/sql.md b/docs/tutorial/flink/sql.md index d6e3d1408..34a97ff85 100644 --- a/docs/tutorial/flink/sql.md +++ b/docs/tutorial/flink/sql.md @@ -1,7 +1,6 @@ The page outlines the steps to manage spatial data using SedonaSQL. ==The example code is written in Java but also works for Scala==. SedonaSQL supports SQL/MM Part3 Spatial SQL Standard. It includes four kinds of SQL operators as follows. All these operators can be directly called through: - ```java Table myTable = tableEnv.sqlQuery("YOUR_SQL") ``` @@ -18,7 +17,6 @@ Detailed SedonaSQL APIs are available here: [SedonaSQL API](../../api/flink/Over ## Initiate Stream Environment Use the following code to initiate your `StreamExecutionEnvironment` at the beginning: - ```java StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); EnvironmentSettings settings = EnvironmentSettings.newInstance().inStreamingMode().build(); @@ -144,7 +142,6 @@ This `ST_Transform` transform the CRS of these geometries from EPSG:4326 to EPSG For example, a Table that has coordinates in the US will become like this. Before the transformation: - ``` +----+--------------------------------+--------------------------------+ | op | geom_point | name_point | @@ -486,7 +483,6 @@ DataStream<Row> geomStream = text.map(new MapFunction<String, Row>() { ### Get Spatial Table Use TableEnv's fromDataStream function, with two column names `geom` and `geom_name`. - ```java Table geomTable = sedona.fromDataStream(geomStream, "geom", "geom_name") ``` diff --git a/docs/tutorial/geopandas-shapely.md b/docs/tutorial/geopandas-shapely.md index a0492bd3b..423f4b3e0 100644 --- a/docs/tutorial/geopandas-shapely.md +++ b/docs/tutorial/geopandas-shapely.md @@ -85,7 +85,6 @@ gdf.plot( ) ``` - <br> <br> diff --git a/docs/tutorial/rdd.md b/docs/tutorial/rdd.md index 27cb25e54..2c231e54e 100644 --- a/docs/tutorial/rdd.md +++ b/docs/tutorial/rdd.md @@ -31,14 +31,12 @@ A generic SpatialRDD is not typed to a certain geometry type and open to more sc Geometries in a WKT and WKB file always occupy a single column no matter how many coordinates they have. Sedona provides `WktReader ` and `WkbReader` to create generic SpatialRDD. Suppose we have a `checkin.tsv` WKT TSV file at Path `/Download/checkin.tsv` as follows: - ``` POINT (-88.331492 32.324142) hotel POINT (-88.175933 32.360763) gas POINT (-88.388954 32.357073) bar POINT (-88.221102 32.35078) restaurant ``` - This file has two columns and corresponding ==offsets==(Column IDs) are 0, 1. Column 0 is the WKT string and Column 1 is the checkin business type. Use the following code to create a SpatialRDD @@ -176,14 +174,11 @@ To create a generic SpatialRDD from CSV, TSV, WKT, WKB and GeoJSON input formats We use checkin.csv CSV file as the example. You can create a generic SpatialRDD using the following steps: 1. Load data in SedonaSQL. - ```scala var df = sedona.read.format("csv").option("header", "false").load(csvPointInputLocation) df.createOrReplaceTempView("inputtable") ``` - 2. Create a Geometry type column in SedonaSQL - ```scala var spatialDf = sedona.sql( """ @@ -191,9 +186,7 @@ var spatialDf = sedona.sql( |FROM inputtable """.stripMargin) ``` - 3. Use SedonaSQL DataFrame-RDD Adapter to convert a DataFrame to an SpatialRDD - ```scala var spatialRDD = Adapter.toSpatialRdd(spatialDf, "checkin") ``` diff --git a/docs/tutorial/snowflake/sql.md b/docs/tutorial/snowflake/sql.md index 88847dfcd..bac45e955 100644 --- a/docs/tutorial/snowflake/sql.md +++ b/docs/tutorial/snowflake/sql.md @@ -21,7 +21,6 @@ FROM city_tbl; ``` Output: - ``` WKT CITY_NAME POINT (-122.33 47.61) Seattle @@ -416,7 +415,6 @@ SELECT ST_AsEWKT(SEDONA.ST_SetSRID(ST_GeometryFromWKT('POINT(1 2)'), 4326)) ``` Output: - ``` SRID=0;POINT(1 2) ``` diff --git a/docs/tutorial/sql.md b/docs/tutorial/sql.md index 83568b80f..a949f66c8 100644 --- a/docs/tutorial/sql.md +++ b/docs/tutorial/sql.md @@ -211,7 +211,6 @@ POLYGON (..., ...) Wahkiakum County POLYGON (..., ...) De Baca County POLYGON (..., ...) Lancaster County ``` - The file may have many other columns. Use the following code to load the data and create a raw DataFrame: @@ -642,7 +641,6 @@ The dataset used here is the Chicago crimes dataset, available [here](https://gi SedonaPyDeck exposes a create_heatmap API which can be used to visualize a heatmap out of the passed SedonaDataFrame containing points: Example: - ```python SedonaPyDeck.create_heatmap(df=crimes_df) ``` diff --git a/docs/tutorial/zeppelin.md b/docs/tutorial/zeppelin.md index 17e5f1abb..e0321a935 100644 --- a/docs/tutorial/zeppelin.md +++ b/docs/tutorial/zeppelin.md @@ -19,7 +19,6 @@ spark.sql( ``` Then create an SQL paragraph to fetch the data - ```sql %sql SELECT * @@ -48,7 +47,6 @@ spark.sql( ``` Then create an SQL paragraph to fetch the data - ```sql %sql SELECT *, 'I am the map center!'
