GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/9375
[SPARK-11188][SQL] Elide stacktraces in bin/spark-sql for AnalysisExceptions
Only print the error message to the console for Analysis Exceptions in
sql-shell
You can merge this pull request
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9348#discussion_r43538406
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeInSuite.scala
---
@@ -82,4 +83,35 @@ class OptimizeInSuite extends
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-152263073
@marmbrus Thanks a LOT for your comments and guidance. I am still little
new to the process.
So do i have to open two JIRAs one for 1.4 and the other
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9348#issuecomment-152305661
@cloud-fan Hi Wenchen, this failure seems unrelated to my change. Should we
retest ?
---
If your project is set up for it, you can reply to this email and have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-152253043
@marmbrus Took out "WIP" and also changed the description.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9348#discussion_r43419241
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizeInSuite.scala
---
@@ -36,6 +36,7 @@ class OptimizeInSuite extends
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9348#discussion_r43354942
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -417,6 +417,14 @@ object NullPropagation extends
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9348#discussion_r43355402
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -417,6 +417,14 @@ object NullPropagation extends
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9348#issuecomment-152422271
@cloud-fan Thanks.. have updated the test case name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/9348
[SPARK-11024][SQL] Optimize NULL in by folding it to
Literal(null)
Add a rule in optimizer to convert NULL [NOT] IN (expr1,...,expr2) to
Literal(null).
This is a follow up defect
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-152001423
@marmbrus
Michael, i will work on your suggestion about having different log4j
default for spark-sql binary in another PR. Thanks a lot for suggesting
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-152000584
@marmbrus
Thank you very much for your comments. I did think about users having their
own log4j. My thinking was that - if they don't specify a console appender
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9194#discussion_r43159249
--- Diff:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
---
@@ -219,4 +219,11 @@ class CliSuite extends
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9194#discussion_r43011192
--- Diff:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
---
@@ -219,4 +219,11 @@ class CliSuite extends
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-150333267
@marmbrus
Michael, i tried putting "retest this please" in the comment but it does
not seem to start the tests ? Is there a way to restart
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-150156202
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-150259879
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9194#issuecomment-150124281
@marmbrus
Michael, need your advice here. It seems like when CliSuite is run, the
tracing is set to console and thats why i see the AnalysisException
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9194#discussion_r42654708
--- Diff:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
---
@@ -308,7 +310,15 @@ private[hive
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/9194
[SPARK-11188][SQL][WIP] Elide stacktraces in bin/spark-sql for
AnalysisExceptions
I have a couple of questions.
1) I have used the a newly added constructor in hive that takes
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-149665013
@marmbrus @rxin
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42092689
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala
---
@@ -135,4 +135,21 @@ class AnalysisSuite extends
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9123#discussion_r42092649
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -482,7 +482,12 @@ class Analyzer(
val
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9123#issuecomment-148314200
@cloud-fan
Hi Wenchen, i have fixed the code based on your comments.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-148102351
@marmbrus
Hi Michael, the tests have passed. Please let me know if anything is
pending before it gets merged.
Thanks a lot.
---
If your project is set up
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/9123
[SPARK-10534][SQL] ORDER BY clause allows only columns that are present in
Sâ¦
Find out the missing attributes by recursively looking
at the sort order expression and rest of the code
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9123#issuecomment-148220723
@cloud-fan
Can you please take a look. Thanks ..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-147537472
@mambrus
Hi Michael, the tests have passed. Please let me know if anything is
pending before it gets merged.
Thanks a lot.
---
If your project is set up
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146470802
@cloud-fan
Thanks a lot. I have implemented the review comments. Please take a look. I
looked at the optimizer code,. we already seem to be transforming NULL
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146685208
@cloud-fan
I didn't see a re-open option on this pull request. Do i have to create a
new pull request ?
Please let me know ..
---
If your project is set
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146692581
@cloud-fan
Hi Wenchen, Can you please review the update to the test case ? Thank you.
---
If your project is set up for it, you can reply to this email
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146703685
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/9036
[SPARK-8654] [SQL] Analysis exception when using NULL IN (...) : invalid
cast
In the analysis phase , while processing the rules for IN predicate, we
compare the in-list types to the lhs
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146696603
@cloud-fan , hi, how do we trigger a retest ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146701712
@cloud-fan
It seems to fail now with the following exception ..
hudson.plugins.git.GitException: Failed to fetch from
https://github.com/apache/spark.git
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146682265
@marmbrus .. sorry about it. Is there a way i can look at the list of
failures ?
I had run :
build/mvn -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver
Github user dilipbiswal commented on a diff in the pull request:
https://github.com/apache/spark/pull/9036#discussion_r41595658
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
---
@@ -304,7 +304,10 @@ object HiveTypeCoercion
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146751654
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146728774
@cloud-fan
Thanks.. Fixed it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146728816
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146756933
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146757252
@cloud-fan
Hi Wenchen, the test failed which looks unrelated. I wanted to retest ..
but looks like its not triggering it. Can you please help ?
---
If your
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/9036#issuecomment-146756112
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146645486
Thanks a lot @marmbrus .
Many thanks to @cloud-fan for his help.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146404345
@cloud-fan
Hi Wenchen, can you please take a look at the changes and let me know what
you think..
---
If your project is set up for it, you can reply
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146287283
Thank you @marmbrus @rick-ibm @cloud-fan
I checked the behavior of db2. It also raises an error if the in list types
are not compatible.
db2
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145948447
@cloud-fan , please confirm my understanding of the code (fairly new to the
codebase..:-)
In the code we go through the entire in list and run evaluate
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145958925
Thanks a LOT @cloud-fan. Sure.. i will look into it. When you say another
PR,
do we mean another JIRA ?
Asking as i am new to the process
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146003985
Very good point..Thanks.. Actually Hive reports an error in this case.
hive> select * from tnull where array(2,3) in (1, array(2,3));
FAI
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-146047895
@marmbrus
Thanks a lot michael for looking into this. I debugged hive to understand
the
behaviour and would like to share my findings. I wanted to make
GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/8983
[SPARK-8654][SQL] Fix Analysis exception when using NULL IN (...)
In the analysis phase , while processing the rules for IN predicate, we
compare the in-list types to the lhs expression type
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145711558
Hi Wenchen,
Here is the link i could find where its a bit confusing on the equality
operator.
https://cwiki.apache.org/confluence/display/Hive
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145713214
hive> select null in (1,2,null) from tnull;
OK
NULL
NULL
NULL
Time taken: 0.139 seconds, Fetched: 3 row(s)
---
If your project is set
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145639934
Thanks for reviewing the code Wenchen. I was trying to model the test case
based on what was put in the JIRA which did a caseInsensitiveAnalyze. I have
fixed it now
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145664718
Thanks !!
Do we need to look at the in list types in this case ? The in list types
could be literals of different types , right ? for example NULL
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145696762
Thanks Wenchen. You are right that not all types can be casted to boolean.
However, in this case, we are not trying to cast the in list types to the
LHS
Github user dilipbiswal commented on the pull request:
https://github.com/apache/spark/pull/8983#issuecomment-145734618
Can you please help clarify ? Are you referring to the case when one of the
value in the
in list is literal null like 1 in (1, NULL) ? If so, i don't think we
1201 - 1257 of 1257 matches
Mail list logo
