Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-03-09 Thread via GitHub
yihua commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1986995077 > @yihua the question may be navie, is a native writer required to cut out the HBase deps eventually? Sorry for the late reply. Yes, we'll also implement native HFile writer.

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-02-22 Thread via GitHub
pan3793 commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1958915909 @yihua the question may be navie, is a native writer required to cut out the HBase deps eventually? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
yihua merged PR #10241: URL: https://github.com/apache/hudi/pull/10241 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
yihua commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1898974246 Azure CI is green. https://github.com/apache/hudi/assets/2497195/348bf86d-0bf5-4159-95e2-3607ad9514e3;> -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1898806710 ## CI report: * 54195223d071f56c572aecee344a516fb41203df Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1898791976 ## CI report: * 54195223d071f56c572aecee344a516fb41203df UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1898349281 ## CI report: * 54195223d071f56c572aecee344a516fb41203df Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-18 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1897986894 ## CI report: * 54195223d071f56c572aecee344a516fb41203df Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1897975748 ## CI report: * 54195223d071f56c572aecee344a516fb41203df UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1897862648 ## CI report: * 54195223d071f56c572aecee344a516fb41203df Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1897705264 ## CI report: * 0e94898672fb426904b8c03c311751fce2ad043a Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1897698970 ## CI report: * 0e94898672fb426904b8c03c311751fce2ad043a Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456726505 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456600994 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java: ## @@ -0,0 +1,124 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456583568 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456574639 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456574639 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456566862 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456562550 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileReaderImpl.java: ## @@ -0,0 +1,284 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456560159 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java: ## @@ -0,0 +1,73 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456552529 ## hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileUtils.java: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456545817 ## hudi-io/README.md: ## @@ -0,0 +1,40 @@ + + +# `hudi-io` Module + +This module contains classes that are I/O related, including common abstraction and APIs, readers

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456539273 ## hudi-io/src/main/java/org/apache/hudi/io/compress/airlift/HoodieAirliftGzipDecompressor.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456532833 ## hudi-io/src/main/java/org/apache/hudi/io/compress/HoodieCompressionFactory.java: ## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1456530802 ## hudi-common/src/test/java/org/apache/hudi/io/storage/TestHoodieHFileReaderWriter.java: ## @@ -474,4 +505,47 @@ private void verifyHFileReader( }

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
vinothchandar commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1454105252 ## hudi-common/src/test/java/org/apache/hudi/io/storage/TestHoodieHFileReaderWriter.java: ## @@ -474,4 +505,47 @@ private void verifyHFileReader( }

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1895661798 ## CI report: * 0e94898672fb426904b8c03c311751fce2ad043a Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1895474130 ## CI report: * 3934af17773124b22860f1d82fe7bb69945e4a9e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1895401931 ## CI report: * 3934af17773124b22860f1d82fe7bb69945e4a9e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-17 Thread via GitHub
yihua commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1895383401 The README is updated with HFile format description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888272284 ## CI report: * 3934af17773124b22860f1d82fe7bb69945e4a9e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888205167 ## CI report: * fcbecfc34daac4a9ec66b71d228862eb213119b1 Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888132678 ## CI report: * 5330999cce0b2b985de3077eb379528910700cd1 Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888092435 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888084909 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888025754 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1888015211 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
yihua commented on code in PR #10241: URL: https://github.com/apache/hudi/pull/10241#discussion_r1449417384 ## hudi-io/src/test/java/org/apache/hudi/io/hfile/TestHFileReader.java: ## @@ -0,0 +1,90 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1887932610 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1887922159 ## CI report: * 064eb14f04ec9a4c58d6952c12908ad4165945bf Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-11 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1887864995 ## CI report: * 78c32e76253bb0db70f289bf88f9560545b5819e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-10 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1886332233 ## CI report: * 78c32e76253bb0db70f289bf88f9560545b5819e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-10 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1886178804 ## CI report: * 78c32e76253bb0db70f289bf88f9560545b5819e Azure:

Re: [PR] [HUDI-7170] Implement HFile reader independent of HBase [hudi]

2024-01-10 Thread via GitHub
hudi-bot commented on PR #10241: URL: https://github.com/apache/hudi/pull/10241#issuecomment-1886172424 ## CI report: * 78c32e76253bb0db70f289bf88f9560545b5819e UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure` re-run