[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-23 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=713493=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713493
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 24/Jan/22 05:10
Start Date: 24/Jan/22 05:10
Worklog Time Spent: 10m 
  Work Description: chitralverma closed pull request #598:
URL: https://github.com/apache/griffin/pull/598


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 713493)
Time Spent: 1h  (was: 50m)

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-23 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=713492=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713492
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 24/Jan/22 05:10
Start Date: 24/Jan/22 05:10
Worklog Time Spent: 10m 
  Work Description: lipzhu opened a new pull request #598:
URL: https://github.com/apache/griffin/pull/598


   **What changes were proposed in this pull request?**
   Built in Avro format is released in Spark 
2.4.0,https://issues.apache.org/jira/browse/SPARK-24768
   For Griffin, we still need to convert the Avro to com.databricks.spark.avro 
in Spark 2.3.x environment. 
   
   **Does this PR introduce any user-facing change?**
   No.
   
   **How was this patch tested?**
   Unit Tests
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 713492)
Time Spent: 50m  (was: 40m)

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-23 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=713491=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713491
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 24/Jan/22 05:06
Start Date: 24/Jan/22 05:06
Worklog Time Spent: 10m 
  Work Description: asfgit closed pull request #598:
URL: https://github.com/apache/griffin/pull/598


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 713491)
Time Spent: 40m  (was: 0.5h)

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-23 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=713489=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713489
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 24/Jan/22 04:50
Start Date: 24/Jan/22 04:50
Worklog Time Spent: 10m 
  Work Description: chitralverma commented on a change in pull request #598:
URL: https://github.com/apache/griffin/pull/598#discussion_r790412145



##
File path: 
measure/src/main/scala/org/apache/griffin/measure/datasource/connector/batch/FileBasedDataConnector.scala
##
@@ -79,7 +79,8 @@ case class FileBasedDataConnector(
 SupportedFormats.contains(format),
 s"Invalid format '$format' specified. Must be one of 
${SupportedFormats.mkString("['", "', '", "']")}")
 
-  if (format.equalsIgnoreCase("avro") && sparkSession.version < "2.3.0") {
+  // Use old implementation for AVRO format if current spark version is not 
2.4.x and above
+  if (format.equalsIgnoreCase("avro") && sparkSession.version < "2.4.0") {

Review comment:
   ```suggestion
 if ("avro".equalsIgnoreCase(format) && sparkSession.version < "2.4.0") {
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 713489)
Time Spent: 0.5h  (was: 20m)

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-23 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=713488=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713488
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 24/Jan/22 04:48
Start Date: 24/Jan/22 04:48
Worklog Time Spent: 10m 
  Work Description: chitralverma commented on a change in pull request #598:
URL: https://github.com/apache/griffin/pull/598#discussion_r790411689



##
File path: 
measure/src/main/scala/org/apache/griffin/measure/datasource/connector/batch/FileBasedDataConnector.scala
##
@@ -79,7 +79,8 @@ case class FileBasedDataConnector(
 SupportedFormats.contains(format),
 s"Invalid format '$format' specified. Must be one of 
${SupportedFormats.mkString("['", "', '", "']")}")
 
-  if (format.equalsIgnoreCase("avro") && sparkSession.version < "2.3.0") {
+  // built-in AVRO data source implementation is released in spark 2.4.0

Review comment:
   Suggestion:
   
   ```suggestion
 // Use old implementation for AVRO format if current spark version is not 
2.4.x and above
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 713488)
Time Spent: 20m  (was: 10m)

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Work logged] (GRIFFIN-369) Bug fix for avro format in data connector

2022-01-18 Thread ASF GitHub Bot (Jira) 


 [ 
https://issues.apache.org/jira/browse/GRIFFIN-369?focusedWorklogId=710482=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-710482
 ]

ASF GitHub Bot logged work on GRIFFIN-369:
--

Author: ASF GitHub Bot
Created on: 18/Jan/22 12:58
Start Date: 18/Jan/22 12:58
Worklog Time Spent: 10m 
  Work Description: lipzhu opened a new pull request #598:
URL: https://github.com/apache/griffin/pull/598


   **What changes were proposed in this pull request?**
   Built in Avro format is released in Spark 
2.4.0,https://issues.apache.org/jira/browse/SPARK-24768
   For Griffin, we still need to convert the Avro to com.databricks.spark.avro 
in Spark 2.3.x environment. 
   
   **Does this PR introduce any user-facing change?**
   No.
   
   **How was this patch tested?**
   Unit Tests
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@griffin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 710482)
Remaining Estimate: 0h
Time Spent: 10m

> Bug fix for avro format in data connector
> -
>
> Key: GRIFFIN-369
> URL: https://issues.apache.org/jira/browse/GRIFFIN-369
> Project: Griffin
>  Issue Type: Bug
>Reporter: Zhu, Lipeng
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> built-in AVRO data source implementation is released in spark 2.4.0. 
> In spark 2.3.x, we need to use 
> com.databricks.spark.avro.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)