[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15087301#comment-15087301 ] mustafa elbehery commented on SPARK-5226: - I have tried to use Aliaksei's implementation on 500MB of GPS Trajectories. The algorithm never finished. Though, his implementation worked very well on the provided sample data. When I have created a scatter plot for both datasets; sample data && trajectories data, I found out that his data's distribution was Gaussian, while mine was very skewed. Moreover, this implementation has a bottleneck, because basically all the partition are merged together in a reduce step, which leads turns the algorithm into Serial again !!!.. I have commented below a better implementation to avoid this bottleneck, hope it would be more helpful. > Add DBSCAN Clustering Algorithm to MLlib > > > Key: SPARK-5226 > URL: https://issues.apache.org/jira/browse/SPARK-5226 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Muhammad-Ali A'rabi >Priority: Minor > Labels: DBSCAN, clustering > > MLlib is all k-means now, and I think we should add some new clustering > algorithms to it. First candidate is DBSCAN as I think. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15067200#comment-15067200 ] mustafa elbehery commented on SPARK-5226: - Better Implementation, based on research paper for parallel DBSCAN can be found here. https://github.com/irvingc/dbscan-on-spark .. The approach solved bottleneck of reduce step, in which discovered clusters are merged. Hope it helps. > Add DBSCAN Clustering Algorithm to MLlib > > > Key: SPARK-5226 > URL: https://issues.apache.org/jira/browse/SPARK-5226 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Muhammad-Ali A'rabi >Priority: Minor > Labels: DBSCAN, clustering > > MLlib is all k-means now, and I think we should add some new clustering > algorithms to it. First candidate is DBSCAN as I think. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mustafa elbehery updated SPARK-5226: Comment: was deleted (was: Hello, I would like to use DBSCAN on spark. [~alitouka] I have tried to use ur implementation, on 500 MG of data. However, I think the **Population of partition index** step is to expensive. Is this implementation is going to be online soon, Regards.) > Add DBSCAN Clustering Algorithm to MLlib > > > Key: SPARK-5226 > URL: https://issues.apache.org/jira/browse/SPARK-5226 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Muhammad-Ali A'rabi >Priority: Minor > Labels: DBSCAN, clustering > > MLlib is all k-means now, and I think we should add some new clustering > algorithms to it. First candidate is DBSCAN as I think. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-5226) Add DBSCAN Clustering Algorithm to MLlib
[ https://issues.apache.org/jira/browse/SPARK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15005355#comment-15005355 ] mustafa elbehery commented on SPARK-5226: - Hello, I would like to use DBSCAN on spark. [~alitouka] I have tried to use ur implementation, on 500 MG of data. However, I think the **Population of partition index** step is to expensive. Is this implementation is going to be online soon, Regards. > Add DBSCAN Clustering Algorithm to MLlib > > > Key: SPARK-5226 > URL: https://issues.apache.org/jira/browse/SPARK-5226 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Muhammad-Ali A'rabi >Priority: Minor > Labels: DBSCAN, clustering > > MLlib is all k-means now, and I think we should add some new clustering > algorithms to it. First candidate is DBSCAN as I think. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
mustafa elbehery created SPARK-11526: Summary: JDBC to PostGIS throws UnSupported Type exception Key: SPARK-11526 URL: https://issues.apache.org/jira/browse/SPARK-11526 Project: Spark Issue Type: Bug Components: SQL Affects Versions: 1.5.1 Environment: Linux Based Reporter: mustafa elbehery I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although the connection works fine with a normal *PostgresSql* Database, it throws UnSupported Type Exception when I try to query a Database with _PostGIS_ extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; {color:red} val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; {color:red} val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; {color:red} val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; {color:red} val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of
[jira] [Comment Edited] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991484#comment-14991484
]
mustafa elbehery edited comment on SPARK-11526 at 11/5/15 10:33 AM:
[~sowen] Correct me if I am wrong, I think the *PostGIS* extension is not
supported in SparkSql JDBC Connector, thats why the returned type is
*unsupported* not "OTHER". I have discussed this issue with [~rams] during
Spark Summit, and he mentioned that it could be fixed.
was (Author: elbehery):
[~sowen] Correct me if I am wrong, I think the *PostGIS* extension is not
supported in SparkSql JDBC Connector, thats why the returned type is
*unsupported* not "OTHER". I have discussed this issue with [~rams] during the
Spark Summit, and he mentioned that it could be fixed.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> from SparkShell as mentioned in SparkSql docs, I had the following results
> :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991484#comment-14991484
]
mustafa elbehery commented on SPARK-11526:
--
[~sowen] Correct me if I am wrong, I think the *PostGIS* extension is not
supported in SparkSql JDBC Connector, thats why the returned type is
*unsupported* not "OTHER". I have discussed this issue with [~rams] during the
Spark Summit, and he mentioned that it could be fixed.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> from SparkShell as mentioned in SparkSql docs, I had the following results
> :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; {color:red} val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; {color:red} val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991460#comment-14991460
]
mustafa elbehery commented on SPARK-11526:
--
I have re-opened the issue
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Priority: Critical (was: Major)
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
*However*, when I tried to use the same way for querying *nycesri*, I have got
*unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work.
was:I have tried to use SparkSQL JDBC to connect to *PostGIS* Database.
Although the connection works fine with a normal *PostgresSql* Database, it
throws UnSupported Type Exception when I try to query a Database with _PostGIS_
extension.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Reopened] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery reopened SPARK-11526:
--
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991497#comment-14991497
]
mustafa elbehery commented on SPARK-11526:
--
Ok, I am sure the problem is not from the code, since the same code was working
with standard database type. Is this means I can not connect to PostGIS through
SparkSQL JDBC ?? .
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> from SparkShell as mentioned in SparkSql docs, I had the following results
> :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991452#comment-14991452
]
mustafa elbehery commented on SPARK-11526:
--
updated
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Comment: was deleted
(was: updated )
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991453#comment-14991453
]
mustafa elbehery commented on SPARK-11526:
--
updated
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Attachment: Selection_007.png
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load()
*However*, when I tried to use the same way for querying *nycesri*, I have got
*unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code; bq. val jdbcDF =
sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load()
I have tried to use PostGIS_JDBC.jar, but it did not work.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
> extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load()
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code; bq. val jdbcDF =
> sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
> I have tried to use PostGIS_JDBC.jar, but it did not work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
the connection works fine with a normal *PostgresSql* Database, it throws
UnSupported Type Exception when I try to query a Database with _PostGIS_
extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports geometry Types and _spatial_ queries,
using _postgis_ extension.
When I tried to use the
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]from
SparkShell as mentioned in SparkSql docs, I had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports geometry Types and _spatial_ queries,
using _postgis_ extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]from
> SparkShell as mentioned in SparkSql docs, I had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
the connection works fine with a normal *PostgresSql* Database, it throws
UnSupported Type Exception when I try to query a Database with _PostGIS_
extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports geometry Types and _spatial_ queries,
using _postgis_ extension.
When I tried to use the
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
from SparkShell as mentioned in SparkSql docs, I had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
the connection works fine with a normal *PostgresSql* Database, it throws
UnSupported Type Exception when I try to query a Database with _PostGIS_
extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports geometry Types and _spatial_ queries,
using _postgis_ extension.
When I tried to use the
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]from
SparkShell as mentioned in SparkSql docs, I had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> from SparkShell as mentioned in SparkSql docs, I had the following results
> :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load()
[jira] [Updated] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mustafa elbehery updated SPARK-11526:
-
Description:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports geometry Types and _spatial_ queries,
using _postgis_ extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
was:
I have tried to use SparkSQL
[JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
to connect to *PostGIS* Database. Although the connection works fine with a
normal *PostgresSql* Database, it throws UnSupported Type Exception when I try
to query a Database with _PostGIS_ extension.
To Further Explain, I have two Databases in my Postgresql, as follows :-
1- *postgres* : a normal database which supports only primitive types.
2- *nycesri* : a database which supports _spatial_ queries, using _postgis_
extension.
When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
had the following results :-
1- with *postgres*, I have retrieved the tables in the DataFrame object, and
could query it using the following code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgresql:postgres",
"dbtable" -> "affiliations")).load() {color}
2- *However*, when I tried to use the same way for querying *nycesri*, I have
got *unsupported Type * exception, probably because the _postGis_ extension
unsupported. Following is the used code;
{color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
Map("url" -> "jdbc:postgis:nycesri",
"dbtable" -> "ny_counties_clip")).load() {color}
I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
screenshot of the exception.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> to connect to *PostGIS* Database. Although the connection works fine with a
> normal *PostgresSql* Database, it throws UnSupported Type Exception when I
> try to query a Database with _PostGIS_ extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the JDBC from SparkShell as mentioned in SparkSql docs, I
> had the following results :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use
[jira] [Commented] (SPARK-11526) JDBC to PostGIS throws UnSupported Type exception
[
https://issues.apache.org/jira/browse/SPARK-11526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991851#comment-14991851
]
mustafa elbehery commented on SPARK-11526:
--
Well, If Spark can support PostGIS JDBC, I think it will work .. Would refer me
where exactly I should start in Spark code ? .. I will try to fix it myself,
and let you know if I could not.
> JDBC to PostGIS throws UnSupported Type exception
> -
>
> Key: SPARK-11526
> URL: https://issues.apache.org/jira/browse/SPARK-11526
> Project: Spark
> Issue Type: Bug
> Components: SQL
>Affects Versions: 1.5.1
> Environment: Linux Based
>Reporter: mustafa elbehery
>Priority: Critical
> Labels: easyfix
> Attachments: Selection_007.png
>
>
> I have tried to use SparkSQL JDBC to connect to *PostGIS* Database. Although
> the connection works fine with a normal *PostgresSql* Database, it throws
> UnSupported Type Exception when I try to query a Database with _PostGIS_
> extension.
> To Further Explain, I have two Databases in my Postgresql, as follows :-
> 1- *postgres* : a normal database which supports only primitive types.
> 2- *nycesri* : a database which supports geometry Types and _spatial_
> queries, using _postgis_ extension.
> When I tried to use the
> [JDBC|http://spark.apache.org/docs/latest/sql-programming-guide.html#jdbc-to-other-databases]
> from SparkShell as mentioned in SparkSql docs, I had the following results
> :-
> 1- with *postgres*, I have retrieved the tables in the DataFrame object, and
> could query it using the following code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgresql:postgres",
> "dbtable" -> "affiliations")).load() {color}
> 2- *However*, when I tried to use the same way for querying *nycesri*, I have
> got *unsupported Type * exception, probably because the _postGis_
> extension unsupported. Following is the used code;
> {color:red} val jdbcDF = sqlContext.read.format("jdbc").options(
> Map("url" -> "jdbc:postgis:nycesri",
> "dbtable" -> "ny_counties_clip")).load() {color}
> I have tried to use PostGIS_JDBC.jar, but it did not work. I have attached a
> screenshot of the exception.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
