Re: Apache Hudi Graduation vote on general@incubator

2020-05-17 Thread 岳伟
+1 Graduate Apache Hudi from the Incubator Harvey Yue On 05/16/2020 22:49,hamid pirahesh wrote: [x ] +1 Graduate Apache Hudi from the Incubator.> On Fri, May 15, 2020 at 7:06 PM Vinoth Chandar wrote: Hello all, Just started the VOTE on the IPMC general list [1] If you are an IPMC

Re: [DISCUSS] should we do a 0.5.3 patch set release ?

2020-05-17 Thread Bhavani Sudha
Hello all, I wanted to send a quick update on 0.5.3 readiness and code freeze. There were few more candidate that requested to go in 0.5.3. Of those, we are still waiting on the following 5 PRs to be reviewed and landed. - #1633 HUDI-858 Allow

Re: Unable to run hudi-cli integration tests

2020-05-17 Thread Pratyaksh Sharma
Hi, For me also the test runs but looking at the error, it looks like no work or deduping is done, which is strange. Here is the error -> [*ERROR*] *Tests **run: 1*, *Failures: 1*, Errors: 0, Skipped: 0, Time elapsed: 8.425 s* <<< FAILURE!* - in org.apache.hudi.cli.integ. *ITTestRepairsCommand*

Re: [Discussion] hudi support log append scenario with better write and asynchronous compaction

2020-05-17 Thread wei li
Thanks, Vinoth Chandar Just like https://issues.apache.org/jira/projects/HUDI/issues/HUDI-112 , we need a mechanism to solve two issues. 1. On the write side: do not compaction for faster write. (now merge on read can solve this problem) 2. compaction and read : also a mechanism to collapse

[ANNOUNCE] Hudi Weekly Update(2020-05-10 ~ 2020-05-17)

2020-05-17 Thread leesf
Dear community, Nice to share Hudi community weekly update for 2020-05-10 ~ 2020-05-17 with updates on graduation, features, discussion, bug fix and tests. === Discussion [Graduation] A voting thread to graduate Hudi to Apache TLP has been started, please

Re: Unable to run hudi-cli integration tests

2020-05-17 Thread hddong
Hi Pratyaksh, Dose it throws the same Exception? And can you check if sparkLauncher throws the same Exception. Most time, ITTest failed due to some config of local spark. I had got this Exception before, but it run successfully after `mvn clean package ...`. Regards hddong Pratyaksh Sharma

Re: Unable to run hudi-cli integration tests

2020-05-17 Thread Shiyan Xu
Hi Pratyaksh, I have the same setup as yours. I would normally tend to clean up my local deps mvn dependency:purge-local-repository mvn clean install -DskipTests -DskipITs mvn -Dtest=ITTestRepairsCommand#testDeduplicateWithReal -DfailIfNoTests=false test Though I was able to run the test, it

Re: Unable to run hudi-cli integration tests

2020-05-17 Thread Pratyaksh Sharma
Hi hddong, Strange but nothing seems to work for me. I tried doing mvn clean and then run travis tests. Also I tried running the command `mvn clean package -DskipTests -DskipITs -Pspark-shade-unbundle-avro` first and then run the test using `mvn -Dtest=ITTestRepairsCommand#testDeduplicateWithReal