[GitHub] [hudi] rswagatika commented on issue #2373: [SUPPORT] - Hudi Read ,Partition Filter is coming empty.

[GitBox]

rswagatika commented on issue #2373:
URL: https://github.com/apache/hudi/issues/2373#issuecomment-751848219


   Hi , I have added the  config for hive style partitioning and in my 
partition I could see the hive style partitons. 
   
   <img width="862" alt="Screen Shot 2020-12-28 at 3 04 16 PM" 
src="https://user-images.githubusercontent.com/76132270/103240306-fe0bd680-491d-11eb-8253-cc9370a166d6.png";>
   
   
   However if I do a filter on read still the Partition Filter is empty in my 
Physical Plan. My question is does hudi scan all the partitioned folder in this 
case or does it only look up the partitioned folder I specified in the read.
   
   == Physical Plan ==
   *(1) Project [_hoodie_commit_time#281, _hoodie_commit_seqno#282, 
_hoodie_record_key#283, _hoodie_partition_path#284, _hoodie_file_name#285, 
locations#286, posting_key#287L, update_date#288, range_partition#289L]
   +- *(1) Filter (((isnotnull(range_partition#289L) && 
isnotnull(posting_key#287L)) && (range_partition#289L = 10)) && 
(posting_key#287L > 20))
      +- *(1) FileScan parquet 
[_hoodie_commit_time#281,_hoodie_commit_seqno#282,_hoodie_record_key#283,_hoodie_partition_path#284,_hoodie_file_name#285,locations#286,posting_key#287L,update_date#288,range_partition#289L]
 Batched: false, Format: Parquet, Location: 
InMemoryFileIndex[s3://bgt-atlas-emr-devtest/testdataprocessing/WOutHive_Partition_test13/.hoodie...,
 PartitionFilters: [], PushedFilters: [IsNotNull(range_partition), 
IsNotNull(posting_key), EqualTo(range_partition,10), GreaterThan(pos..., 
ReadSchema: 
struct<_hoodie_commit_time:string,_hoodie_commit_seqno:string,_hoodie_record_key:string,_hoodie_p...


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org