[GitHub] spark issue #16997: Updated the Spark SQL Programming guide with Custom obje...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 Sure, and thanks for kind attention to this pull request. Thank You Best Regards | *Harsh Sharma* Sr. Software Consultant Knoldus Software LLP FB <https://www.facebook.com/harsh.sharma.161446> | Twitter <https://twitter.com/harsh_sharma5> | LinkedIn <https://www.linkedin.com/in/harsh-sharma-0a08a1b0?trk=hp-identity-name> harshs...@gmail.com Skype*: khandal60* *+91-8447307237* On Sun, Mar 5, 2017 at 10:13 PM, Sean Owen <notificati...@github.com> wrote: > This still has formatting and text problems. I'm sorry I don't think I can > go around again for this when it's not an important change, and I'd like to > close this. > > â > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/16997#issuecomment-284242129>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AKIiQARgsS9c8P7s7slP6T39bwCfW7ywks5riuZGgaJpZM4MF0vf> > . > --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17058: Refactored code to remove null representation
Github user HarshSharma8 closed the pull request at: https://github.com/apache/spark/pull/17058 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17054: Refactored the code to remove redundency of count...
Github user HarshSharma8 closed the pull request at: https://github.com/apache/spark/pull/17054 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17058: Refactored code to remove null representation
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/17058 May be this is right and readable from your point of view, this code is returning nothing else, just null again and again as default value if user along its other values are not present. From my point of view i just let the user know about the default value that i am going to use. On Feb 24, 2017 6:04 PM, "Hyukjin Kwon" <notificati...@github.com> wrote: > It seems too tirival. The default seems even always null. I don't think > this is an improvement if there is no other specific reason. The original > codes are more readable to me. > > â > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/17058#issuecomment-282280760>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AKIiQL-Ob-HDj0Jz8ykRo_7frY7qFhT1ks5rfs42gaJpZM4MLI9I> > . > --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17058: Refactored code to remove null representation
GitHub user HarshSharma8 opened a pull request: https://github.com/apache/spark/pull/17058 Refactored code to remove null representation ## What changes were proposed in this pull request? Removed the null value that was returned directly with null representation and returned in a better way. ## How was this patch tested? The result remains same, its just a refactoring of code. Please review http://spark.apache.org/contributing.html before opening a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HarshSharma8/spark refactor/removeNull Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/17058.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17058 commit 11cbd9d5f58e19f04743daffcbcd985327079817 Author: Harsh Sharma <ha...@knoldus.com> Date: 2017-02-24T12:08:12Z Refactored code to remove null representation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #17054: Refactored the code to remove redundency of count...
GitHub user HarshSharma8 opened a pull request: https://github.com/apache/spark/pull/17054 Refactored the code to remove redundency of count operation ## What changes were proposed in this pull request? Removed the redundant count operation which is generating same result when it not required to be performed twice. ## How was this patch tested? Its already a duplicate operation to be performed, so its already tested. Please review http://spark.apache.org/contributing.html before opening a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HarshSharma8/spark remove/redundency Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/17054.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #17054 commit 14785f52e5f4048ea687e97e7044b3de00716d89 Author: Harsh Sharma <ha...@knoldus.com> Date: 2017-02-24T07:15:14Z Refactored the code to remove redundency of count operation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16997: Updated the Spark SQL Programming guide with Custom obje...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 Did anyone get a chance to verify it or any changes required by me to make ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16997: Updated the Spark SQL Programming guide with Custom obje...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 Hello HyukjinKwon, I have updated the title, i wish you like it, it shows what is there in the content. And commit has already been made. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16997: Updated the SQL programming guide to explain about the E...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 I updated the content with a demo object. I would appreciate if anyone can have a look at this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16997: Updated the SQL programming guide to explain about the E...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 Hello Sean, I have updated the content with back-ticks, Can you have a look at this ? And i am not getting which object-name you are asking about. Thank You Best Regards | *Harsh Sharma* Sr. Software Consultant Facebook <https://www.facebook.com/harsh.sharma.161446> | Twitter <https://twitter.com/harsh_sharma5> | Linked In <https://www.linkedin.com/in/harsh-sharma-0a08a1b0?trk=hp-identity-name> harshs...@gmail.com Skype*: khandal60* *+91-8447307237* On Tue, Feb 21, 2017 at 11:03 AM, Harsh Sharma <ha...@knoldus.com> wrote: > Hello Sean, > I apologize for bold instead of back-ticks, and i'm updating the content > for this. > > > Thank You > > > Best Regards | > *Harsh Sharma* > Sr. Software Consultant > Facebook <https://www.facebook.com/harsh.sharma.161446> | Twitter > <https://twitter.com/harsh_sharma5> | Linked In > <https://www.linkedin.com/in/harsh-sharma-0a08a1b0?trk=hp-identity-name> > harshs...@gmail.com > Skype*: khandal60* > *+91-8447307237* > > On Tue, Feb 21, 2017 at 10:58 AM, Sean Owen <notificati...@github.com> > wrote: > >> *@srowen* commented on this pull request. >> -- >> >> In docs/sql-programming-guide.md >> <https://github.com/apache/spark/pull/16997#discussion_r102134397>: >> >> > @@ -297,6 +297,9 @@ reflection and become the names of the columns. Case classes can also be nested >> types such as `Seq`s or `Array`s. This RDD can be implicitly converted to a DataFrame and then be >> registered as a table. Tables can be used in subsequent SQL statements. >> >> +Spark Encoders are used to convert a JVM object to Spark SQL representation. To create dataset, spark requires an encoder which takes the form of Encoder[T] where T is the type which has to be encoded. Creation of a dataset with a custom type of object, may result into java.lang.UnsupportedOperationException: No Encoder found for Object-Name. >> >> Yes, @HarshSharma8 <https://github.com/HarshSharma8> this still doesn't >> address the comments. Use back-ticks for code, not bold, too. What is >> Object-Name? >> >> â >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub >> <https://github.com/apache/spark/pull/16997#discussion_r102134397>, or mute >> the thread >> <https://github.com/notifications/unsubscribe-auth/AKIiQM8Tsz96c1KHGszvbFmgJnnRD62Gks5renYPgaJpZM4MF0vf> >> . >> > > --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16997: Updated the SQL programming guide to explain about the E...
Github user HarshSharma8 commented on the issue: https://github.com/apache/spark/pull/16997 Hello Sean, I apologize for bold instead of back-ticks, and i'm updating the content for this. Thank You Best Regards | *Harsh Sharma* Sr. Software Consultant Facebook <https://www.facebook.com/harsh.sharma.161446> | Twitter <https://twitter.com/harsh_sharma5> | Linked In <https://www.linkedin.com/in/harsh-sharma-0a08a1b0?trk=hp-identity-name> harshs...@gmail.com Skype*: khandal60* *+91-8447307237* On Tue, Feb 21, 2017 at 10:58 AM, Sean Owen <notificati...@github.com> wrote: > *@srowen* commented on this pull request. > -- > > In docs/sql-programming-guide.md > <https://github.com/apache/spark/pull/16997#discussion_r102134397>: > > > @@ -297,6 +297,9 @@ reflection and become the names of the columns. Case classes can also be nested > types such as `Seq`s or `Array`s. This RDD can be implicitly converted to a DataFrame and then be > registered as a table. Tables can be used in subsequent SQL statements. > > +Spark Encoders are used to convert a JVM object to Spark SQL representation. To create dataset, spark requires an encoder which takes the form of Encoder[T] where T is the type which has to be encoded. Creation of a dataset with a custom type of object, may result into java.lang.UnsupportedOperationException: No Encoder found for Object-Name. > > Yes, @HarshSharma8 <https://github.com/HarshSharma8> this still doesn't > address the comments. Use back-ticks for code, not bold, too. What is > Object-Name? > > â > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/16997#discussion_r102134397>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AKIiQM8Tsz96c1KHGszvbFmgJnnRD62Gks5renYPgaJpZM4MF0vf> > . > --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16997: Updated the SQL programming guide to explain abou...
Github user HarshSharma8 commented on a diff in the pull request: https://github.com/apache/spark/pull/16997#discussion_r102011677 --- Diff: docs/sql-programming-guide.md --- @@ -297,6 +297,9 @@ reflection and become the names of the columns. Case classes can also be nested types such as `Seq`s or `Array`s. This RDD can be implicitly converted to a DataFrame and then be registered as a table. Tables can be used in subsequent SQL statements. +Spark Encoders are used to convert a JVM object to Spark SQL representation. When we want to make a datase, Spark requires an encoder which takes the form Encoder[T] where T is the type we want to be encoded. When we try to create dataset with a custom type of object, then may result into java.lang.UnsupportedOperationException: No Encoder found for Object-Name. --- End diff -- Hello srowen, I have updated the content to match the void of the content, you can have another look at it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16997: Updated the SQL programming guide to explain abou...
GitHub user HarshSharma8 opened a pull request: https://github.com/apache/spark/pull/16997 Updated the SQL programming guide to explain about the Encoding opera⦠## What changes were proposed in this pull request? Made some updates to SQL programming guide to explain the Encoding operation with kryo. ## How was this patch tested? Just updated the docs. Please review http://spark.apache.org/contributing.html before opening a pull request. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HarshSharma8/spark feature/docs Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/16997.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16997 commit 103906fb23b5212858e89e9a090693b6fb2c6307 Author: Harsh Sharma <ha...@knoldus.com> Date: 2017-02-20T06:51:55Z Updated the SQL programming guide to explain about the Encoding operation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
