[jira] [Commented] (LIVY-569) BinaryClassificationMetrics give AttributeError

2019-11-07 Thread Yiheng Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/LIVY-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969212#comment-16969212
 ] 

Yiheng Wang commented on LIVY-569:
--

>From the latest community version of livy, I cannot reprocude the error.

Maybe you need to check with AWS EMR guys...

> BinaryClassificationMetrics give AttributeError
> ---
>
> Key: LIVY-569
> URL: https://issues.apache.org/jira/browse/LIVY-569
> Project: Livy
>  Issue Type: Bug
>Reporter: Fred de Gier
>Priority: Major
>
> When using the BinaryClassificationMetrics there is an error, possible 
> related to conversion of data types. This only occurs when using Sparkmagic 
> and Livy.
> *Input:*
> {code:java}
> from pyspark import SparkContext 
>  from pyspark.sql import SparkSession 
>  import pyspark
>  from pyspark.mllib.evaluation import BinaryClassificationMetrics
> a = sc.parallelize([
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
>  ])
> metrics = BinaryClassificationMetrics(a)
> {code}
>  
> *Output:*
> {code:java}
> 'StructField' object has no attribute '_get_object_id' {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (LIVY-569) BinaryClassificationMetrics give AttributeError

2019-11-01 Thread Fred de Gier (Jira) 


[ 
https://issues.apache.org/jira/browse/LIVY-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965093#comment-16965093
 ] 

Fred de Gier commented on LIVY-569:
---

No, we were using Hortonworks. I believe Spark 2.3 and latest version (at the 
time) of Livy. I asked my old colleagues to follow up on this ticket, they will 
be able to provide the correct version numbers.

> BinaryClassificationMetrics give AttributeError
> ---
>
> Key: LIVY-569
> URL: https://issues.apache.org/jira/browse/LIVY-569
> Project: Livy
>  Issue Type: Bug
>Reporter: Fred de Gier
>Priority: Major
>
> When using the BinaryClassificationMetrics there is an error, possible 
> related to conversion of data types. This only occurs when using Sparkmagic 
> and Livy.
> *Input:*
> {code:java}
> from pyspark import SparkContext 
>  from pyspark.sql import SparkSession 
>  import pyspark
>  from pyspark.mllib.evaluation import BinaryClassificationMetrics
> a = sc.parallelize([
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
>  ])
> metrics = BinaryClassificationMetrics(a)
> {code}
>  
> *Output:*
> {code:java}
> 'StructField' object has no attribute '_get_object_id' {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (LIVY-569) BinaryClassificationMetrics give AttributeError

2019-10-31 Thread Yiheng Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/LIVY-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964538#comment-16964538
 ] 

Yiheng Wang commented on LIVY-569:
--


>From the comments of the stack overflow question, it looks like it's related 
>with AWS EMR Livy. Are you also using the AWS EMR livy?

> BinaryClassificationMetrics give AttributeError
> ---
>
> Key: LIVY-569
> URL: https://issues.apache.org/jira/browse/LIVY-569
> Project: Livy
>  Issue Type: Bug
>Reporter: Fred de Gier
>Priority: Major
>
> When using the BinaryClassificationMetrics there is an error, possible 
> related to conversion of data types. This only occurs when using Sparkmagic 
> and Livy.
> *Input:*
> {code:java}
> from pyspark import SparkContext 
>  from pyspark.sql import SparkSession 
>  import pyspark
>  from pyspark.mllib.evaluation import BinaryClassificationMetrics
> a = sc.parallelize([
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
>  ])
> metrics = BinaryClassificationMetrics(a)
> {code}
>  
> *Output:*
> {code:java}
> 'StructField' object has no attribute '_get_object_id' {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (LIVY-569) BinaryClassificationMetrics give AttributeError

2019-10-30 Thread Fred de Gier (Jira) 


[ 
https://issues.apache.org/jira/browse/LIVY-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963413#comment-16963413
 ] 

Fred de Gier commented on LIVY-569:
---

Hi, I'm no longer working for the company where we had this issue, but other 
people reported the same issue:

https://stackoverflow.com/questions/53939145/structfield-object-has-no-attribute-get-object-id

> BinaryClassificationMetrics give AttributeError
> ---
>
> Key: LIVY-569
> URL: https://issues.apache.org/jira/browse/LIVY-569
> Project: Livy
>  Issue Type: Bug
>Reporter: Fred de Gier
>Priority: Major
>
> When using the BinaryClassificationMetrics there is an error, possible 
> related to conversion of data types. This only occurs when using Sparkmagic 
> and Livy.
> *Input:*
> {code:java}
> from pyspark import SparkContext 
>  from pyspark.sql import SparkSession 
>  import pyspark
>  from pyspark.mllib.evaluation import BinaryClassificationMetrics
> a = sc.parallelize([
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
>  ])
> metrics = BinaryClassificationMetrics(a)
> {code}
>  
> *Output:*
> {code:java}
> 'StructField' object has no attribute '_get_object_id' {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (LIVY-569) BinaryClassificationMetrics give AttributeError

2019-10-30 Thread Yiheng Wang (Jira) 


[ 
https://issues.apache.org/jira/browse/LIVY-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962882#comment-16962882
 ] 

Yiheng Wang commented on LIVY-569:
--

Hi, which version of spark/livy are you using. I cannot reproduce this error. 
Please check the attachment.

 !屏幕快照 2019-10-30 下午6.03.04.png! 

> BinaryClassificationMetrics give AttributeError
> ---
>
> Key: LIVY-569
> URL: https://issues.apache.org/jira/browse/LIVY-569
> Project: Livy
>  Issue Type: Bug
>Reporter: Fred de Gier
>Priority: Major
>
> When using the BinaryClassificationMetrics there is an error, possible 
> related to conversion of data types. This only occurs when using Sparkmagic 
> and Livy.
> *Input:*
> {code:java}
> from pyspark import SparkContext 
>  from pyspark.sql import SparkSession 
>  import pyspark
>  from pyspark.mllib.evaluation import BinaryClassificationMetrics
> a = sc.parallelize([
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
>  (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
>  (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
>  ])
> metrics = BinaryClassificationMetrics(a)
> {code}
>  
> *Output:*
> {code:java}
> 'StructField' object has no attribute '_get_object_id' {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)