[arrow-datafusion-python] branch main updated: Introduces utility for obtaining SqlTable information from a file like location (#398)

[agrove]

This is an automated email from the ASF dual-hosted git repository.

agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion-python.git


The following commit(s) were added to refs/heads/main by this push:
     new 93f8063  Introduces utility for obtaining SqlTable information from a 
file like location (#398)
93f8063 is described below

commit 93f8063f5a0d6058b3a3406ab87c84426b37a3b7
Author: Jeremy Dyer <jdy...@gmail.com>
AuthorDate: Mon Jul 24 08:46:26 2023 -0400

    Introduces utility for obtaining SqlTable information from a file like 
location (#398)
    
    * checkpoint commit
    
    * Introduce BaseSessionContext abstract class
    
    * Introduce abstract methods for CRUD schema operations
    
    * Clean up schema.rs file
    
    * Introduce CRUD methods for table instances
    
    * Add function to drop_table
    
    * Add schema_name to drop_table function
    
    * remove unused parameter in SqlTable new
    
    * Update function to allow for modifying existing tables
    
    * Add functionality for generating SqlTable information from input sources
    
    * Add functionality for generating SqlTable information from input sources
    
    * Adding a utility method to convert arrow type strings to DataType 
instances
    
    * Add method to DataTypeMap for getting the DataType from an Arrow type 
string instance
    
    * Adjust pytests
    
    * Add back deprecated int96 parquet datatype
---
 datafusion/input/location.py | 1 -
 src/common/data_type.rs      | 2 +-
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/datafusion/input/location.py b/datafusion/input/location.py
index 1214075..efbc82f 100644
--- a/datafusion/input/location.py
+++ b/datafusion/input/location.py
@@ -50,7 +50,6 @@ class LocationInputPlugin(BaseInputSource):
             num_rows = metadata.num_rows
 
             # Iterate through the schema and build the SqlTable
-            print(f"Metadata Schema: {metadata.schema}")
             for col in metadata.schema:
                 columns.append(
                     (
diff --git a/src/common/data_type.rs b/src/common/data_type.rs
index fe5f6f0..199fb61 100644
--- a/src/common/data_type.rs
+++ b/src/common/data_type.rs
@@ -321,7 +321,7 @@ impl DataTypeMap {
             "int32" => Ok(DataType::Int32),
             "int64" => Ok(DataType::Int64),
             "int96" => {
-                // Int96 is an old datatype that is now deprecated. We convert 
to nanosecond timestamp
+                // Int96 is an old parquet datatype that is now deprecated. We 
convert to nanosecond timestamp
                 Ok(DataType::Timestamp(TimeUnit::Nanosecond, None))
             }
             "float" => Ok(DataType::Float32),