[GitHub] [hudi] xiarixiaoyao commented on pull request #3330: [HUDI-2101][RFC-28]support z-order for hudi

Mon, 08 Nov 2021 17:43:08 -0800 


xiarixiaoyao commented on pull request #3330:
URL: https://github.com/apache/hudi/pull/3330#issuecomment-963735674


   @xushiyan   
   Test result for z-order
   query resource  --executor-memory 12G --executor-cores 4 --num-executors 4
   <html>
   <body>
   <!--StartFragment--><span 
data-shimo-docs="[[30,[{&quot;A1&quot;:[40,[[20,&quot;Table\n&quot;]]],&quot;B1&quot;:[40,[[20,&quot;File
 nums\n&quot;]]],&quot;C1&quot;:[40,[[20,&quot;Data 
size\n&quot;]]],&quot;D1&quot;:[40,[[20,&quot;1st 
query\n&quot;]]],&quot;E1&quot;:[40,[[20,&quot;2st 
query\n&quot;]]],&quot;F1&quot;:[40,[[20,&quot;3st query (enable   data 
skipping)\n&quot;]]],&quot;G1&quot;:[40,[[20,&quot;4st query (enable   data 
skipping)\n&quot;]]],&quot;H1&quot;:[40,[[20,&quot;DataSkipping\n&quot;]]],&quot;A2&quot;:[40,[[20,&quot;Hudi
 origin 
table\n&quot;]]],&quot;B2&quot;:[40,[[20,&quot;2w\n&quot;]]],&quot;C2&quot;:[40,[[20,&quot;10billion\n&quot;]]],&quot;D2&quot;:[40,[[20,&quot;446806ms\n&quot;]]],&quot;E2&quot;:[40,[[20,&quot;397131
 ms\n&quot;]]],&quot;F2&quot;:[40,[[20,&quot;not 
support\n&quot;]]],&quot;G2&quot;:[40,[[20,&quot;not 
support\n&quot;],[20,&quot;&nbsp;&quot;,&quot;7:1&quot;],[20,&quot;\n&quot;]]],&quot;H2&quot;:[40,[[20,&quot;&nbsp;\n&quot;]]],&quot;A3&quot;:[40,
 [[20,&quot;z-order with two   column 
sorting\n&quot;]]],&quot;B3&quot;:[40,[[20,&quot;2w\n&quot;]]],&quot;C3&quot;:[40,[[20,&quot;10billion\n&quot;]]],&quot;D3&quot;:[40,[[20,&quot;94104
 ms\n&quot;]]],&quot;E3&quot;:[40,[[20,&quot;80778 
ms\n&quot;]]],&quot;F3&quot;:[40,[[20,&quot;5913 
ms\n&quot;]]],&quot;G3&quot;:[40,[[20,&quot;2769ms\n&quot;]]],&quot;H3&quot;:[40,[[20,&quot;Total
 file   size is: 20000, after file skip size is: 5 skipping percent 
0.99975\n&quot;]]],&quot;A4&quot;:[40,[[20,&quot;z-order with four-column   
sorting\n&quot;]]],&quot;B4&quot;:[40,[[20,&quot;2w\n&quot;]]],&quot;C4&quot;:[40,[[20,&quot;10billion\n&quot;]]],&quot;D4&quot;:[40,[[20,&quot;111393
 ms\n&quot;]]],&quot;E4&quot;:[40,[[20,&quot;91664 
ms\n&quot;]]],&quot;F4&quot;:[40,[[20,&quot;5868 
ms\n&quot;]]],&quot;G4&quot;:[40,[[20,&quot;3186 
ms\n&quot;]]],&quot;H4&quot;:[40,[[20,&quot;Total file   size is: 20000, after 
file skip size is: 5 skipping percent 
0.99975\n&quot;]]]},[[10,1,&quot;4:58.5795&quot;],[10,
 
1,&quot;4:51.392&quot;],[10,2,&quot;4:115.852&quot;]],[[10,1,&quot;3:76&quot;],[10,7,&quot;3:77&quot;]]],&quot;25:\&quot;X1gKsn\&quot;&quot;]]">
   
   Table | File nums | Data size | 1st query | 2st query | 3st query (enable   
data skipping) | 4st query (enable   data skipping) | DataSkipping
   -- | -- | -- | -- | -- | -- | -- | --
   Hudi origin table | 2w | 10billion | 446806ms | 397131 ms | not support | 
not support |  
   z-order with two   column sorting | 2w | 10billion | 94104 ms | 80778 ms | 
5913 ms | 2769ms | Total file   size is: 20000, after file skip size is: 5 
skipping percent 0.99975
   z-order with four-column   sorting | 2w | 10billion | 111393 ms | 91664 ms | 
5868 ms | 3186 ms | Total file   size is: 20000, after file skip size is: 5 
skipping percent 0.99975
   
   </span><!--EndFragment-->
   </body>
   </html>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to