GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/18580
[SPARK-21354] [SQL] INPUT FILE related functions do not support more than
one sources
### What changes were proposed in this pull request?
The build-in functions `input_file_name`, `input_file_block_start`,
`input_file_block_length` do not support more than one sources, like what Hive
does. Currently, Spark does not block it and the outputs are
ambiguous/non-deterministic. It could be from any side.
```
hive> select *, INPUT__FILE__NAME FROM t1, t2;
FAILED: SemanticException Column INPUT__FILE__NAME Found in more than One
Tables/Subqueries
```
This PR blocks it and issues an error.
### How was this patch tested?
Added a test case
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark inputFileName
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18580.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18580
----
commit 04ff406c5a82f9454419d0f0054b1aa75ea97aa0
Author: gatorsmile <[email protected]>
Date: 2017-07-09T22:52:55Z
fix.
commit 596ea17bc99a703004ab7bef657603a9db57d5f2
Author: gatorsmile <[email protected]>
Date: 2017-07-09T23:02:36Z
fix.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
