Github user dongjoon-hyun commented on the issue:

    https://github.com/apache/spark/pull/18580
  
    The following is the output of the current Spark.
    ```scala
    scala> spark.range(10).write.saveAsTable("t1")
    
    scala> spark.range(100,110).write.saveAsTable("t2")
    
    scala> sql("select *, input_file_name() from t1").show(false)
    
+---+-------------------------------------------------------------------------------------------------------------------+
    |id |input_file_name()                                                      
                                            |
    
+---+-------------------------------------------------------------------------------------------------------------------+
    |3  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00003-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |4  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00003-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |8  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00007-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |9  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00007-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |0  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00000-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |1  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00001-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |2  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00002-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |5  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00004-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |6  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00005-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |7  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00006-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    
+---+-------------------------------------------------------------------------------------------------------------------+
    
    
    scala> sql("select *, input_file_name() from t2").show(false)
    
+---+-------------------------------------------------------------------------------------------------------------------+
    |id |input_file_name()                                                      
                                            |
    
+---+-------------------------------------------------------------------------------------------------------------------+
    
|103|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00003-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|104|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00003-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|108|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00007-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|109|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00007-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|100|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00000-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|101|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00001-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|102|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00002-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|105|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00004-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|106|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00005-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|107|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00006-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
+---+-------------------------------------------------------------------------------------------------------------------+
    
    
    scala> sql("select *, input_file_name() from ((select * from t1) union all 
(select * from t2))").show(false)
    
+---+-------------------------------------------------------------------------------------------------------------------+
    |id |input_file_name()                                                      
                                            |
    
+---+-------------------------------------------------------------------------------------------------------------------+
    |3  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00003-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |4  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00003-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |8  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00007-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |9  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00007-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |0  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00000-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |1  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00001-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |2  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00002-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |5  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00004-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |6  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00005-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    |7  
|file:///Users/dongjoon/spark/spark-warehouse/t1/part-00006-b0ca8fa4-03ae-4e3a-b4b4-a13d601cd155-c000.snappy.parquet|
    
|103|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00003-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|104|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00003-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|108|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00007-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|109|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00007-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|100|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00000-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|101|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00001-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|102|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00002-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|105|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00004-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|106|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00005-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
|107|file:///Users/dongjoon/spark/spark-warehouse/t2/part-00006-76ea547d-0187-40f0-b5dd-f9f1fffeeabf-c000.snappy.parquet|
    
+---+-------------------------------------------------------------------------------------------------------------------+
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to