Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-05-31 Thread via GitHub
codope commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1622387730 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieFileGroupReaderRecordReader.java: ## @@ -0,0 +1,294 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-05-21 Thread via GitHub
codope commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1607649124 ## hudi-common/src/main/java/org/apache/hudi/common/engine/HoodieReaderContext.java: ## @@ -123,10 +129,23 @@ public abstract ClosableIterator getFileRecordIterator(

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-20 Thread via GitHub
jonvex commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1954508175 > Overall looks good to me. @jonvex : What Hive versions are we targeting/testing ? @bvaradar I used the docker demo to test. I think that is using Hive 2. We would like this to

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-20 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1496057861 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-20 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1496053647 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-20 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1496049414 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-11 Thread via GitHub
bvaradar commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1485731251 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,259 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-07 Thread via GitHub
jonvex commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1932549046 Thanks @bvaradar, we all appreciate it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-06 Thread via GitHub
bvaradar commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1931349128 > @bvaradar Can you help the review of the hive related code? Yes @danny0405 . Will review this PR. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-03 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1477164731 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-02 Thread via GitHub
danny0405 commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1925026965 @bvaradar Can you help the review of the hive related code? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-02 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1476626353 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieFileGroupReaderRecordReader.java: ## @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-02-02 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1476613616 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471439782 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471420556 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-30 Thread via GitHub
xiarixiaoyao commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1471121556 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1470524689 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469933163 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecordMerger.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469931251 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecord.java: ## @@ -0,0 +1,236 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469930915 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecord.java: ## @@ -0,0 +1,236 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + *

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469919029 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469248006 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469244004 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieFileGroupReaderRecordReader.java: ## @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-29 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469244004 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieFileGroupReaderRecordReader.java: ## @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469185474 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
xicm commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469185474 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieParquetInputFormat.java: ## @@ -91,9 +94,42 @@ private void initAvroInputFormat() { } } + private

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469035209 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/ObjectInspectorCache.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469033287 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecordMerger.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469032745 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecord.java: ## @@ -0,0 +1,236 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469032673 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieHiveRecord.java: ## @@ -0,0 +1,236 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469031742 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HoodieFileGroupReaderRecordReader.java: ## @@ -0,0 +1,279 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469030424 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469029597 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-28 Thread via GitHub
danny0405 commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1469029149 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/HiveHoodieReaderContext.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-19 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1460108785 ## hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeRecordReader.java: ## @@ -116,6 +117,7 @@ public void setUp() {

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-19 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1901370527 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 15ed1ad17c8b99804d6e404342a11fab6e212935 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-19 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1901057461 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 8f08fa5850559a29da9aca426d74e188daf995f5 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-19 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1901044565 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 8f08fa5850559a29da9aca426d74e188daf995f5 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-19 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1459475819 ## hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/realtime/AbstractRealtimeRecordReader.java: ## @@ -101,6 +102,9 @@ public

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-15 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1893068084 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * 8f08fa5850559a29da9aca426d74e188daf995f5 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-15 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1892965427 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * c0fbf8d4c4492880cc60f659fdba825df5395900 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-15 Thread via GitHub
hudi-bot commented on PR #10422: URL: https://github.com/apache/hudi/pull/10422#issuecomment-1892960266 ## CI report: * 99517e23baa60a6a0602e9daf7f522f3c1dcfa1e UNKNOWN * c0fbf8d4c4492880cc60f659fdba825df5395900 Azure:

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-09 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1446299229 ## hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeRecordReader.java: ## @@ -116,6 +117,7 @@ public void setUp() {

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-09 Thread via GitHub
jonvex commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1446284069 ## packaging/bundle-validation/validate.sh: ## @@ -93,7 +93,7 @@ test_spark_hadoop_mr_bundles () { # save HiveQL query results

Re: [PR] [WIP] [HUDI-6787] Implement the HoodieFileGroupReader API for Hive [hudi]

2024-01-08 Thread via GitHub
vinothchandar commented on code in PR #10422: URL: https://github.com/apache/hudi/pull/10422#discussion_r1445060501 ## hudi-hadoop-mr/src/test/java/org/apache/hudi/hadoop/realtime/TestHoodieRealtimeRecordReader.java: ## @@ -116,6 +117,7 @@ public void setUp() {