[GitHub] [incubator-hudi] yanghua commented on issue #991: Hudi Test Suite (Refactor)

[GitBox]

yanghua commented on issue #991: Hudi Test Suite (Refactor) 
URL: https://github.com/apache/incubator-hudi/pull/991#issuecomment-560272068
 
 
   Some pending job from my side:
   
   * introduce `DistributedTestDataSource` to refactor the existed 
`DeltaGenerator`;
   * rename `hudi-bench` module name to other name;
   * replace all the code style: `log.info(String.format(xxx));`;
   * research Azure Pipelines;


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Assigned] (HUDI-209) Implement JMX metrics reporter

[Forward Xu (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Forward Xu reassigned HUDI-209:
---

Assignee: Forward Xu

> Implement JMX metrics reporter
> --
>
> Key: HUDI-209
> URL: https://issues.apache.org/jira/browse/HUDI-209
> Project: Apache Hudi (incubating)
>  Issue Type: New Feature
>Reporter: vinoyang
>Assignee: Forward Xu
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.1
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently, there are only two reporters {{MetricsGraphiteReporter}} and 
> {{InMemoryMetricsReporter}}. {{InMemoryMetricsReporter}} is used for testing. 
> So actually we only have one metrics reporter. Since JMX is a standard of the 
> monitor on the JVM platform, I propose to provide a JMX metrics reporter. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #116

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 2.21 KB...]
m2.conf
mvn
mvn.cmd
mvnDebug
mvnDebug.cmd
mvnyjp

/home/jenkins/tools/maven/apache-maven-3.5.4/boot:
plexus-classworlds-2.5.2.jar

/home/jenkins/tools/maven/apache-maven-3.5.4/conf:
logging
settings.xml
toolchains.xml

/home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging:
simplelogger.properties

/home/jenkins/tools/maven/apache-maven-3.5.4/lib:
aopalliance-1.0.jar
cdi-api-1.0.jar
cdi-api.license
commons-cli-1.4.jar
commons-cli.license
commons-io-2.5.jar
commons-io.license
commons-lang3-3.5.jar
commons-lang3.license
ext
guava-20.0.jar
guice-4.2.0-no_aop.jar
jansi-1.17.1.jar
jansi-native
javax.inject-1.jar
jcl-over-slf4j-1.7.25.jar
jcl-over-slf4j.license
jsr250-api-1.0.jar
jsr250-api.license
maven-artifact-3.5.4.jar
maven-artifact.license
maven-builder-support-3.5.4.jar
maven-builder-support.license
maven-compat-3.5.4.jar
maven-compat.license
maven-core-3.5.4.jar
maven-core.license
maven-embedder-3.5.4.jar
maven-embedder.license
maven-model-3.5.4.jar
maven-model-builder-3.5.4.jar
maven-model-builder.license
maven-model.license
maven-plugin-api-3.5.4.jar
maven-plugin-api.license
maven-repository-metadata-3.5.4.jar
maven-repository-metadata.license
maven-resolver-api-1.1.1.jar
maven-resolver-api.license
maven-resolver-connector-basic-1.1.1.jar
maven-resolver-connector-basic.license
maven-resolver-impl-1.1.1.jar
maven-resolver-impl.license
maven-resolver-provider-3.5.4.jar
maven-resolver-provider.license
maven-resolver-spi-1.1.1.jar
maven-resolver-spi.license
maven-resolver-transport-wagon-1.1.1.jar
maven-resolver-transport-wagon.license
maven-resolver-util-1.1.1.jar
maven-resolver-util.license
maven-settings-3.5.4.jar
maven-settings-builder-3.5.4.jar
maven-settings-builder.license
maven-settings.license
maven-shared-utils-3.2.1.jar
maven-shared-utils.license
maven-slf4j-provider-3.5.4.jar
maven-slf4j-provider.license
org.eclipse.sisu.inject-0.3.3.jar
org.eclipse.sisu.inject.license
org.eclipse.sisu.plexus-0.3.3.jar
org.eclipse.sisu.plexus.license
plexus-cipher-1.7.jar
plexus-cipher.license
plexus-component-annotations-1.7.1.jar
plexus-component-annotations.license
plexus-interpolation-1.24.jar
plexus-interpolation.license
plexus-sec-dispatcher-1.4.jar
plexus-sec-dispatcher.license
plexus-utils-3.1.0.jar
plexus-utils.license
slf4j-api-1.7.25.jar
slf4j-api.license
wagon-file-3.1.0.jar
wagon-file.license
wagon-http-3.1.0-shaded.jar
wagon-http.license
wagon-provider-api-3.1.0.jar
wagon-provider-api.license

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/ext:
README.txt

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native:
freebsd32
freebsd64
linux32
linux64
osx
README.txt
windows32
windows64

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/freebsd32:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/freebsd64:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/linux32:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/linux64:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/osx:
libjansi.jnilib

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/windows32:
jansi.dll

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/windows64:
jansi.dll
Finished /home/jenkins/tools/maven/apache-maven-3.5.4 Directory Listing :
Detected current version as: 
'HUDI_home=
0.5.1-SNAPSHOT'
[INFO] Scanning for projects...
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Hudi   [pom]
[INFO] hudi-common[jar]
[INFO] hudi-timeline-service  [jar]
[INFO] hudi-hadoop-mr [jar]
[INFO] hudi-client[jar]
[INFO] hudi-hive  [jar]
[INFO] hudi-spark [jar]
[INFO] hudi-utilities [jar]
[INFO] hudi-cli   [jar]
[INFO] hudi-hadoop-mr-bundle  [jar]
[INFO] hudi-hive-bundle   [jar]
[INFO] hudi-spark-bundle  [jar]
[INFO] hudi-presto-bundle [jar]
[INFO] hudi-utilities-bundle  [jar]
[INFO] hudi-timeline-server-bundle[jar]
[INFO] hudi-hadoop-docker 

[GitHub] [incubator-hudi] lamber-ken opened a new pull request #1066: [checkstyle] Add ConstantName java checkstyle rule

[GitBox]

lamber-ken opened a new pull request #1066: [checkstyle] Add ConstantName java 
checkstyle rule
URL: https://github.com/apache/incubator-hudi/pull/1066
 
 
   ## What is the purpose of the pull request
   
   Add ConstantName java checkstyle rule
   
   ## Brief change log
   
 - Refactor based on new ConstantName code style rule
   
   ## Verify this pull request
   
   This pull request is code cleanup without any test coverage.
   
   ## Committer checklist
   
- [ ] Has a corresponding JIRA in PR title & commit

- [ ] Commit message is descriptive of the change

- [ ] CI is green
   
- [ ] Necessary doc changes done or have another open PR
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Closed] (HUDI-374) Unable to generateUpdates in QuickstartUtils

[leesf (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

leesf closed HUDI-374.
--
Fix Version/s: 0.5.1
   Resolution: Fixed

Fixed via master: b65a897856259e7872e39a9e3e68661926592d7b

> Unable to generateUpdates in QuickstartUtils
> 
>
> Key: HUDI-374
> URL: https://issues.apache.org/jira/browse/HUDI-374
> Project: Apache Hudi (incubating)
>  Issue Type: Bug
>  Components: Spark datasource
>Reporter: hong dongdong
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.1
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> {code:java}
> scala> convertToStringList(dataGen.generateInserts(1))
> res0: java.util.List[String] = [{"ts": 0.0, "uuid": 
> "78956d3a-c13b-4871-8b14-596b2a7e11d9", "rider": "rider-213", "driver": 
> "driver-213", "begin_lat": 0.4726905879569653, "begin_lon": 
> 0.46157858450465483, "end_lat": 0.754803407008858, "end_lon": 
> 0.9671159942018241, "fare": 34.158284716382845, "partitionpath": 
> "americas/brazil/sao_paulo"}]
> scala> convertToStringList(dataGen.generateUpdates(1))
> java.lang.IllegalArgumentException: bound must be positive
>   at java.util.Random.nextInt(Random.java:388)
>   at 
> org.apache.hudi.QuickstartUtils$DataGenerator.generateUpdates(QuickstartUtils.java:163)
>   ... 73 elided
> {code}
> When `numExistingKeys = 1`, `rand.nextInt(numExistingKeys - 1 )`  is 
> equivalent to `rand.nextInt(0)` and bound of nextInt() must be  positive.  On 
> the other hand,  the range of nextInt is [0, numExistingKeys),  
> rand.nextInt(numExistingKeys) here is right.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (HUDI-357) Refactor hudi-cli based on new comment and code style rules

[leesf (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

leesf closed HUDI-357.
--
Fix Version/s: 0.5.1
   Resolution: Fixed

Fixed via master: 75132c139f0faf9ef68bb461b3a551238a377455

> Refactor hudi-cli based on new comment and code style rules
> ---
>
> Key: HUDI-357
> URL: https://issues.apache.org/jira/browse/HUDI-357
> Project: Apache Hudi (incubating)
>  Issue Type: Sub-task
>Reporter: Gurudatt Kulkarni
>Assignee: Gurudatt Kulkarni
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.1
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> This issue used to refactor hudi-cli module based on new comments and code 
> style rules.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Closed] (HUDI-370) Refactor hudi-common based on new ImportOrder code style rule

[leesf (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

leesf closed HUDI-370.
--
Resolution: Fixed

Fixed via master: 784e3ad0b65ce8f7e0de2e2935032fb2573d5a26

> Refactor hudi-common based on new ImportOrder code style rule
> -
>
> Key: HUDI-370
> URL: https://issues.apache.org/jira/browse/HUDI-370
> Project: Apache Hudi (incubating)
>  Issue Type: Sub-task
>Reporter: lamber-ken
>Assignee: lamber-ken
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Refactor hudi-common based on new ImportOrder code style rule



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [incubator-hudi] leesf merged pull request #1063: [HUDI-370] Refactor hudi-common based on new ImportOrder code style rule

[GitBox]

leesf merged pull request #1063: [HUDI-370] Refactor hudi-common based on new 
ImportOrder code style rule
URL: https://github.com/apache/incubator-hudi/pull/1063
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] vinothchandar merged pull request #1065: [DOCS] Add EMR to powered by page

[GitBox]

vinothchandar merged pull request #1065: [DOCS] Add EMR to powered by page
URL: https://github.com/apache/incubator-hudi/pull/1065
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[incubator-hudi] branch asf-site updated: [DOCS] Add EMR to powered by page (#1065)

[vinoth]

This is an automated email from the ASF dual-hosted git repository.

vinoth pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 5966142  [DOCS] Add EMR to powered by page (#1065)
5966142 is described below

commit 5966142539be95ca20635a2f54a943ac3fb8bae8
Author: vinoth chandar 
AuthorDate: Sun Dec 1 07:26:30 2019 -0800

[DOCS] Add EMR to powered by page (#1065)
---
 docs/powered_by.md | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/docs/powered_by.md b/docs/powered_by.md
index 50e657c..165d0b5 100644
--- a/docs/powered_by.md
+++ b/docs/powered_by.md
@@ -10,9 +10,13 @@ toc: false
 
  Uber
 
-Hudi was originally developed at [Uber](https://uber.com), to achieve [low 
latency database ingestion, with high 
efficiency](http://www.slideshare.net/vinothchandar/hadoop-strata-talk-uber-your-hadoop-has-arrived/32).
-It has been in production since Aug 2016, powering ~100 highly business 
critical tables on Hadoop, worth 100s of TBs(including top 10 including 
trips,riders,partners).
-It also powers several incremental Hive ETL pipelines and being currently 
integrated into Uber's data dispersal system.
+Apache Hudi was originally developed at [Uber](https://uber.com), to achieve 
[low latency database ingestion, with high 
efficiency](http://www.slideshare.net/vinothchandar/hadoop-strata-talk-uber-your-hadoop-has-arrived/32).
+It has been in production since Aug 2016, powering the massive [100PB data 
lake](https://eng.uber.com/uber-big-data-platform/), including highly business 
critical tables like core trips,riders,partners. It also 
+powers several incremental Hive ETL pipelines and being currently integrated 
into Uber's data dispersal system.
+
+ Amazon EMR
+Amazon Web Services is the World's leading cloud services provider. Apache 
Hudi is [pre-installed](https://aws.amazon.com/emr/features/hudi/) with the AWS 
Elastic Map Reduce 
+offering, providing means for AWS users to perform record-level 
updates/deletes and manage storage efficiently.
 
  EMIS Health
 

[GitHub] [incubator-hudi] vinothchandar opened a new pull request #1065: [DOCS] Add EMR to powered by page

[GitBox]

vinothchandar opened a new pull request #1065: [DOCS] Add EMR to powered by page
URL: https://github.com/apache/incubator-hudi/pull/1065
 
 
   ## *Tips*
   - *Thank you very much for contributing to Apache Hudi.*
   - *Please review https://hudi.apache.org/contributing.html before opening a 
pull request.*
   
   ## What is the purpose of the pull request
   
   Updating the powered by page
   
   ## Brief change log
   
   
   ## Verify this pull request
   
   This pull request is a trivial rework / code cleanup without any test 
coverage.
   
   ## Committer checklist
   
- [ ] Has a corresponding JIRA in PR title & commit

- [ ] Commit message is descriptive of the change

- [ ] CI is green
   
- [ ] Necessary doc changes done or have another open PR
  
- [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services