[jira] [Assigned] (HUDI-414) Refactor handling of layout version filters in active timeline

[leesf (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

leesf reassigned HUDI-414:
--

Assignee: leesf

> Refactor handling of layout version filters in active timeline
> --
>
> Key: HUDI-414
> URL: https://issues.apache.org/jira/browse/HUDI-414
> Project: Apache Hudi (incubating)
>  Issue Type: Task
>Reporter: Balaji Varadarajan
>Assignee: leesf
>Priority: Minor
>
> Per code-review comment : 
> [https://github.com/apache/incubator-hudi/pull/1009#discussion_r357181383]
> One idea is to introduce factory methods with name explicitly suggesting if 
> instants are filtered or not ? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HUDI-391) Rename module name from hudi-bench to hudi-test-suite

[vinoyang (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

vinoyang resolved HUDI-391.
---
Resolution: Done

Done via hudi_test_suite_refactor branch : 
80ca4ddf28a648ca3296d03f073f09eb559ebdac

> Rename module name from hudi-bench to hudi-test-suite
> -
>
> Key: HUDI-391
> URL: https://issues.apache.org/jira/browse/HUDI-391
> Project: Apache Hudi (incubating)
>  Issue Type: Sub-task
>  Components: Testing
>Reporter: vinoyang
>Assignee: vinoyang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[incubator-hudi] branch hudi_test_suite_refactor updated (eaaf3f6 -> ae5bd06)

[vinoyang]

This is an automated email from the ASF dual-hosted git repository.

vinoyang pushed a change to branch hudi_test_suite_refactor
in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git.


from eaaf3f6  [HUDI-394] Provide a basic implementation of test suite
 add ae5bd06  [HUDI-391] Rename module name from hudi-bench to 
hudi-test-suite and fix some checkstyle issues (#1102)

No new revisions were added by this update.

Summary of changes:
 docker/hoodie/hadoop/hive_base/Dockerfile  |  2 +-
 docker/hoodie/hadoop/hive_base/pom.xml |  4 ++--
 {hudi-bench => hudi-test-suite}/pom.xml|  2 +-
 .../prepare_integration_suite.sh   |  4 ++--
 .../hudi/testsuite}/DFSDeltaWriterAdapter.java |  6 +++---
 .../hudi/testsuite}/DFSSparkAvroDeltaWriter.java   |  8 
 .../apache/hudi/testsuite}/DeltaInputFormat.java   |  2 +-
 .../apache/hudi/testsuite}/DeltaOutputType.java|  2 +-
 .../apache/hudi/testsuite}/DeltaWriterAdapter.java |  4 ++--
 .../apache/hudi/testsuite}/DeltaWriterFactory.java | 10 -
 .../testsuite}/configuration/DFSDeltaConfig.java   |  6 +++---
 .../hudi/testsuite}/configuration/DeltaConfig.java |  6 +++---
 .../hudi/testsuite}/converter/UpdateConverter.java |  6 +++---
 .../org/apache/hudi/testsuite}/dag/DagUtils.java   |  8 
 .../hudi/testsuite}/dag/ExecutionContext.java  |  9 
 .../apache/hudi/testsuite}/dag/WorkflowDag.java|  4 ++--
 .../hudi/testsuite}/dag/WorkflowDagGenerator.java  | 12 +--
 .../hudi/testsuite}/dag/nodes/BulkInsertNode.java  |  6 +++---
 .../hudi/testsuite}/dag/nodes/CleanNode.java   |  4 ++--
 .../hudi/testsuite}/dag/nodes/CompactNode.java |  6 +++---
 .../apache/hudi/testsuite}/dag/nodes/DagNode.java  |  6 +++---
 .../hudi/testsuite}/dag/nodes/HiveQueryNode.java   |  8 
 .../hudi/testsuite}/dag/nodes/HiveSyncNode.java|  8 
 .../hudi/testsuite}/dag/nodes/InsertNode.java  | 10 -
 .../hudi/testsuite}/dag/nodes/RollbackNode.java|  6 +++---
 .../testsuite}/dag/nodes/ScheduleCompactNode.java  |  6 +++---
 .../testsuite}/dag/nodes/SparkSQLQueryNode.java|  8 
 .../hudi/testsuite}/dag/nodes/UpsertNode.java  |  8 
 .../hudi/testsuite}/dag/nodes/ValidateNode.java|  6 +++---
 .../testsuite}/dag/scheduler/DagScheduler.java | 12 +--
 .../hudi/testsuite}/generator/DeltaGenerator.java  | 24 +++---
 .../FlexibleSchemaRecordGenerationIterator.java|  2 +-
 .../GenericRecordFullPayloadGenerator.java |  2 +-
 .../GenericRecordFullPayloadSizeEstimator.java |  2 +-
 .../GenericRecordPartialPayloadGenerator.java  |  2 +-
 .../generator/LazyRecordGeneratorIterator.java |  2 +-
 .../generator/UpdateGeneratorIterator.java |  2 +-
 .../helpers/DFSTestSuitePathSelector.java  |  2 +-
 .../testsuite}/helpers/HiveServiceProvider.java|  6 +++---
 .../testsuite}/job/HoodieDeltaStreamerWrapper.java |  2 +-
 .../hudi/testsuite}/job/HoodieTestSuiteJob.java| 24 +++---
 .../testsuite}/reader/DFSAvroDeltaInputReader.java |  8 
 .../testsuite}/reader/DFSDeltaInputReader.java |  2 +-
 .../reader/DFSHoodieDatasetInputReader.java|  2 +-
 .../reader/DFSParquetDeltaInputReader.java |  6 +++---
 .../hudi/testsuite}/reader/DeltaInputReader.java   |  2 +-
 .../hudi/testsuite}/reader/SparkBasedReader.java   |  2 +-
 .../testsuite}/writer/AvroDeltaInputWriter.java|  2 +-
 .../hudi/testsuite}/writer/DeltaInputWriter.java   |  2 +-
 .../apache/hudi/testsuite}/writer/DeltaWriter.java |  6 +++---
 .../testsuite}/writer/FileDeltaInputWriter.java|  2 +-
 .../writer/SparkAvroDeltaInputWriter.java  |  2 +-
 .../apache/hudi/testsuite}/writer/WriteStats.java  |  2 +-
 .../hudi/testsuite}/TestDFSDeltaWriterAdapter.java | 18 
 .../hudi/testsuite}/TestFileDeltaInputWriter.java  | 14 ++---
 .../configuration/TestWorkflowBuilder.java | 12 +--
 .../testsuite}/converter/TestUpdateConverter.java  |  4 ++--
 .../apache/hudi/testsuite}/dag/TestComplexDag.java | 12 +--
 .../apache/hudi/testsuite}/dag/TestDagUtils.java   | 10 -
 .../hudi/testsuite}/dag/TestHiveSyncDag.java   | 12 +--
 .../hudi/testsuite}/dag/TestInsertOnlyDag.java |  8 
 .../hudi/testsuite}/dag/TestInsertUpsertDag.java   | 10 -
 .../TestGenericRecordPayloadEstimator.java |  6 +++---
 .../TestGenericRecordPayloadGenerator.java | 14 ++---
 .../generator/TestWorkloadGenerator.java   | 22 ++--
 .../testsuite}/job/TestHoodieTestSuiteJob.java | 22 ++--
 .../reader/TestDFSAvroDeltaInputReader.java|  4 ++--
 .../reader/TestDFSHoodieDatasetInputReader.java|  4 ++--
 .../apache/hudi/testsuite}/utils/TestUtils.java|  2 +-
 .../hudi/testsuite}/writer/TestDeltaWriter.java|  6 +++---
 

[GitHub] [incubator-hudi] yanghua merged pull request #1102: [HUDI-391] Rename module name from hudi-bench to hudi-test-suite

[GitBox]

yanghua merged pull request #1102: [HUDI-391] Rename module name from 
hudi-bench to hudi-test-suite
URL: https://github.com/apache/incubator-hudi/pull/1102
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] akshaythakur1112 commented on issue #143: Tracking ticket for folks to be added to slack group

[GitBox]

akshaythakur1112 commented on issue #143: Tracking ticket for folks to be added 
to slack group
URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-565920211
 
 
   Please add me:
   akshay.amu.1...@gmail.com


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] akshaythakur1112 removed a comment on issue #143: Tracking ticket for folks to be added to slack group

[GitBox]

akshaythakur1112 removed a comment on issue #143: Tracking ticket for folks to 
be added to slack group
URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-565919089
 
 
   Please add me:
   akshay.si...@pharmeasy.in


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] akshaythakur1112 commented on issue #143: Tracking ticket for folks to be added to slack group

[GitBox]

akshaythakur1112 commented on issue #143: Tracking ticket for folks to be added 
to slack group
URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-565919089
 
 
   Please add me:
   akshay.si...@pharmeasy.in


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] hddong closed pull request #1098: [HUDI-401]Remove unnecessary use of spark in savepoint timeline

[GitBox]

hddong closed pull request #1098: [HUDI-401]Remove unnecessary use of spark in 
savepoint timeline
URL: https://github.com/apache/incubator-hudi/pull/1098
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Updated] (HUDI-308) Avoid Renames for tracking state transitions of all actions on dataset

[Balaji Varadarajan (Jira)]

 [ 
https://issues.apache.org/jira/browse/HUDI-308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Balaji Varadarajan updated HUDI-308:

Status: Closed  (was: Patch Available)

> Avoid Renames for tracking state transitions of all actions on dataset
> --
>
> Key: HUDI-308
> URL: https://issues.apache.org/jira/browse/HUDI-308
> Project: Apache Hudi (incubating)
>  Issue Type: Improvement
>  Components: Common Core
>Reporter: Balaji Varadarajan
>Assignee: Balaji Varadarajan
>Priority: Major
>  Labels: pull-request-available
> Fix For: 0.5.1
>
> Attachments: IMG_0118.jpg
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, We employ renames when transitioning states (REQUESTED, INFLIGHT, 
> COMPLETED) of all actions in Hudi. 
> The idea is to always create new files pertaining to each state of an action 
> (commit, compaction, clean, ) that is being performed and have the 
> Timeline management resolve conflicts when loading them from .hoodie to 
> folder.  The Archiving logic will cleanup transient state files and archive 
> terminal state files. 
> THis handling will be done consistently for all kinds of actions on datasets. 
> As part of this project, we will cleanup un-necessary fields in metada, 
> version them and standardize on avro/json.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [incubator-hudi] bvaradar merged pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar merged pull request #1009:  [HUDI-308] Avoid Renames for tracking 
state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358057459
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/TimelineLayout.java
 ##
 @@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.table;
+
+import org.apache.hudi.common.model.TimelineLayoutVersion;
+import org.apache.hudi.common.table.timeline.HoodieInstant;
+import org.apache.hudi.common.util.collection.Pair;
+
+import java.io.Serializable;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+
+/**
+ * Timeline Layout responsible for applying specific filters when generating 
timeline instants.
+ */
+public abstract class TimelineLayout implements Serializable {
+
+  private static final Map LAYOUT_MAP = 
new HashMap<>();
+
+  static {
+LAYOUT_MAP.put(new TimelineLayoutVersion(TimelineLayoutVersion.VERSION_0), 
new TimelineLayoutV0());
+LAYOUT_MAP.put(new TimelineLayoutVersion(TimelineLayoutVersion.VERSION_1), 
new TimelineLayoutV1());
+  }
+
+  public static TimelineLayout getLayout(TimelineLayoutVersion version) {
+return LAYOUT_MAP.get(version);
+  }
+
+  public abstract Stream 
filterHoodieInstants(Stream instantStream);
+
+  /**
+   * Table Layout where state transitions are managed by renaming files.
+   */
+  private static class TimelineLayoutV0 extends TimelineLayout {
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] vinothchandar commented on issue #1098: [HUDI-401]Remove unnecessary use of spark in savepoint timeline

[GitBox]

vinothchandar commented on issue #1098: [HUDI-401]Remove unnecessary use of 
spark in savepoint timeline
URL: https://github.com/apache/incubator-hudi/pull/1098#issuecomment-565907200
 
 
   @yanghua will be leading that effort.. Can we close this PR for now and 
consider this more holistically.. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #130

[Apache Jenkins Server]

See 


Changes:


--
[...truncated 2.15 KB...]
/home/jenkins/tools/maven/apache-maven-3.5.4/bin:
m2.conf
mvn
mvn.cmd
mvnDebug
mvnDebug.cmd
mvnyjp

/home/jenkins/tools/maven/apache-maven-3.5.4/boot:
plexus-classworlds-2.5.2.jar

/home/jenkins/tools/maven/apache-maven-3.5.4/conf:
logging
settings.xml
toolchains.xml

/home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging:
simplelogger.properties

/home/jenkins/tools/maven/apache-maven-3.5.4/lib:
aopalliance-1.0.jar
cdi-api-1.0.jar
cdi-api.license
commons-cli-1.4.jar
commons-cli.license
commons-io-2.5.jar
commons-io.license
commons-lang3-3.5.jar
commons-lang3.license
ext
guava-20.0.jar
guice-4.2.0-no_aop.jar
jansi-1.17.1.jar
jansi-native
javax.inject-1.jar
jcl-over-slf4j-1.7.25.jar
jcl-over-slf4j.license
jsr250-api-1.0.jar
jsr250-api.license
maven-artifact-3.5.4.jar
maven-artifact.license
maven-builder-support-3.5.4.jar
maven-builder-support.license
maven-compat-3.5.4.jar
maven-compat.license
maven-core-3.5.4.jar
maven-core.license
maven-embedder-3.5.4.jar
maven-embedder.license
maven-model-3.5.4.jar
maven-model-builder-3.5.4.jar
maven-model-builder.license
maven-model.license
maven-plugin-api-3.5.4.jar
maven-plugin-api.license
maven-repository-metadata-3.5.4.jar
maven-repository-metadata.license
maven-resolver-api-1.1.1.jar
maven-resolver-api.license
maven-resolver-connector-basic-1.1.1.jar
maven-resolver-connector-basic.license
maven-resolver-impl-1.1.1.jar
maven-resolver-impl.license
maven-resolver-provider-3.5.4.jar
maven-resolver-provider.license
maven-resolver-spi-1.1.1.jar
maven-resolver-spi.license
maven-resolver-transport-wagon-1.1.1.jar
maven-resolver-transport-wagon.license
maven-resolver-util-1.1.1.jar
maven-resolver-util.license
maven-settings-3.5.4.jar
maven-settings-builder-3.5.4.jar
maven-settings-builder.license
maven-settings.license
maven-shared-utils-3.2.1.jar
maven-shared-utils.license
maven-slf4j-provider-3.5.4.jar
maven-slf4j-provider.license
org.eclipse.sisu.inject-0.3.3.jar
org.eclipse.sisu.inject.license
org.eclipse.sisu.plexus-0.3.3.jar
org.eclipse.sisu.plexus.license
plexus-cipher-1.7.jar
plexus-cipher.license
plexus-component-annotations-1.7.1.jar
plexus-component-annotations.license
plexus-interpolation-1.24.jar
plexus-interpolation.license
plexus-sec-dispatcher-1.4.jar
plexus-sec-dispatcher.license
plexus-utils-3.1.0.jar
plexus-utils.license
slf4j-api-1.7.25.jar
slf4j-api.license
wagon-file-3.1.0.jar
wagon-file.license
wagon-http-3.1.0-shaded.jar
wagon-http.license
wagon-provider-api-3.1.0.jar
wagon-provider-api.license

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/ext:
README.txt

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native:
freebsd32
freebsd64
linux32
linux64
osx
README.txt
windows32
windows64

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/freebsd32:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/freebsd64:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/linux32:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/linux64:
libjansi.so

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/osx:
libjansi.jnilib

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/windows32:
jansi.dll

/home/jenkins/tools/maven/apache-maven-3.5.4/lib/jansi-native/windows64:
jansi.dll
Finished /home/jenkins/tools/maven/apache-maven-3.5.4 Directory Listing :
Detected current version as: 
'HUDI_home=
0.5.1-SNAPSHOT'
[INFO] Scanning for projects...
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Hudi   [pom]
[INFO] hudi-common[jar]
[INFO] hudi-timeline-service  [jar]
[INFO] hudi-hadoop-mr [jar]
[INFO] hudi-client[jar]
[INFO] hudi-hive  [jar]
[INFO] hudi-spark [jar]
[INFO] hudi-utilities [jar]
[INFO] hudi-cli   [jar]
[INFO] hudi-hadoop-mr-bundle  [jar]
[INFO] hudi-hive-bundle   [jar]
[INFO] hudi-spark-bundle  [jar]
[INFO] hudi-presto-bundle [jar]
[INFO] hudi-utilities-bundle  [jar]
[INFO] hudi-timeline-server-bundle

[GitHub] [incubator-hudi] hddong commented on issue #1098: [HUDI-401]Remove unnecessary use of spark in savepoint timeline

[GitBox]

hddong commented on issue #1098: [HUDI-401]Remove unnecessary use of spark in 
savepoint timeline
URL: https://github.com/apache/incubator-hudi/pull/1098#issuecomment-565874413
 
 
   @vinothchandar yep, flink integration is a good idea.I think it would't be 
an issue after spark divided to a specific module. Have any plan for it? maybe 
i also can contribute some code.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] vinothchandar commented on issue #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

vinothchandar commented on issue #1009:  [HUDI-308] Avoid Renames for tracking 
state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#issuecomment-565867921
 
 
   @bvaradar Feel free to merge when you feel this is ready


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] vinothchandar commented on a change in pull request #976: [HUDI-106] Adding support for DynamicBloomFilter

[GitBox]

vinothchandar commented on a change in pull request #976: [HUDI-106] Adding 
support for DynamicBloomFilter
URL: https://github.com/apache/incubator-hudi/pull/976#discussion_r358021740
 
 

 ##
 File path: 
hudi-common/src/test/java/org/apache/hudi/common/bloom/filter/TestInternalDynamicBloomFilter.java
 ##
 @@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.bloom.filter;
+
+import org.apache.hadoop.util.hash.Hash;
+import org.junit.Test;
+
+/**
+ * Unit tests {@link InternalDynamicBloomFilter} for size bounding.
+ */
+public class TestInternalDynamicBloomFilter {
+
+  @Test
+  public void testBoundedSize() {
+
+int[] batchSizes = {1000, 1, 1, 10, 10, 1};
+int indexForMaxGrowth = 3;
+int maxSize = batchSizes[0] * 100;
+BloomFilter filter = new HoodieDynamicBoundedBloomFilter(batchSizes[0], 
0.01, Hash.MURMUR_HASH, maxSize);
+int index = 0;
+int lastKnownBloomSize = 0;
+while (index < batchSizes.length) {
+  for (int i = 0; i < batchSizes[index]; i++) {
+String key = 
org.apache.commons.lang.RandomStringUtils.randomAlphanumeric(50);
+filter.add(key);
+  }
+
+  String serString = filter.serializeToString();
+  if (index != 0) {
+int curLength = serString.length();
+if (index > indexForMaxGrowth) {
+  assert curLength == lastKnownBloomSize;
 
 Review comment:
   could you please use the junit assert methods? Also this is better written 
as a parameterized test? (like the bloom filter test; leave it you to make the 
final call)


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (HUDI-414) Refactor handling of layout version filters in active timeline

[Balaji Varadarajan (Jira)]

Balaji Varadarajan created HUDI-414:
---

 Summary: Refactor handling of layout version filters in active 
timeline
 Key: HUDI-414
 URL: https://issues.apache.org/jira/browse/HUDI-414
 Project: Apache Hudi (incubating)
  Issue Type: Task
Reporter: Balaji Varadarajan


Per code-review comment : 
[https://github.com/apache/incubator-hudi/pull/1009#discussion_r357181383]

One idea is to introduce factory methods with name explicitly suggesting if 
instants are filtered or not ? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358011854
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java
 ##
 @@ -414,23 +436,41 @@ public String getCommitActionType() {
   /**
* Helper method to scan all hoodie-instant metafiles and construct 
HoodieInstant objects.
*
-   * @param fs FileSystem
-   * @param metaPath Meta Path where hoodie instants are present
* @param includedExtensions Included hoodie extensions
+   * @param applyLayoutVersionFilters Depending on Timeline layout version, if 
there are multiple states for the same
+   * action instant, only include the highest state
* @return List of Hoodie Instants generated
* @throws IOException in case of failure
*/
-  public static List 
scanHoodieInstantsFromFileSystem(FileSystem fs, Path metaPath,
-  Set includedExtensions) throws IOException {
-return Arrays.stream(HoodieTableMetaClient.scanFiles(fs, metaPath, path -> 
{
-  // Include only the meta files with extensions that needs to be included
-  String extension = FSUtils.getFileExtension(path.getName());
-  return includedExtensions.contains(extension);
-})).sorted(Comparator.comparing(
-// Sort the meta-data by the instant time (first part of the file name)
-fileStatus -> FSUtils.getInstantTime(fileStatus.getPath().getName(
-// create HoodieInstantMarkers from FileStatus, which extracts 
properties
-.map(HoodieInstant::new).collect(Collectors.toList());
+  public List scanHoodieInstantsFromFileSystem(Set 
includedExtensions,
+  boolean applyLayoutVersionFilters) throws IOException {
+return scanHoodieInstantsFromFileSystem(new Path(metaPath), 
includedExtensions, applyLayoutVersionFilters);
+  }
+
+  /**
+   * Helper method to scan all hoodie-instant metafiles and construct 
HoodieInstant objects.
+   *
+   * @param timelinePath MetaPath where instant files are stored
+   * @param includedExtensions Included hoodie extensions
+   * @param applyLayoutVersionFilters Depending on Timeline layout version, if 
there are multiple states for the same
+   * action instant, only include the highest state
+   * @return List of Hoodie Instants generated
+   * @throws IOException in case of failure
+   */
+  public List scanHoodieInstantsFromFileSystem(Path 
timelinePath, Set includedExtensions,
+  boolean applyLayoutVersionFilters) throws IOException {
+Stream instantStream = Arrays.stream(
+HoodieTableMetaClient
+.scanFiles(getFs(), timelinePath, path -> {
+  // Include only the meta files with extensions that needs to be 
included
+  String extension = FSUtils.getFileExtension(path.getName());
+  return includedExtensions.contains(extension);
+})).map(HoodieInstant::new);
+
+if (applyLayoutVersionFilters) {
+  instantStream = 
TimelineLayout.getLayout(getTimelineLayoutVersion()).filterHoodieInstants(instantStream);
 
 Review comment:
   The key case here is for archival where we need all instants without 
filtering. May be introduce couple of factory methods which instantiate 
HoodieActiveTimeline w/o filtering ?  HUDI-414


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358009213
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java
 ##
 @@ -112,6 +116,10 @@ public static void createHoodieProperties(FileSystem fs, 
Path metadataFolder, Pr
   if (!properties.containsKey(HOODIE_ARCHIVELOG_FOLDER_PROP_NAME)) {
 properties.setProperty(HOODIE_ARCHIVELOG_FOLDER_PROP_NAME, 
DEFAULT_ARCHIVELOG_FOLDER);
   }
+  if (!properties.containsKey(HOODIE_TIMELINE_LAYOUT_VERSION)) {
 
 Review comment:
   The version in hoodie.properties is treated as default configuration. 
MetaClient Writers override this config with version-1 (unless they opt out). 
MetaClient readers will always use latest version (version - 1) which is 
backwards compatible. Keeping a version in hoodie.properties would also 
hopefully be useful in future migrations


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] nsivabalan edited a comment on issue #976: [HUDI-106] Adding support for DynamicBloomFilter

[GitBox]

nsivabalan edited a comment on issue #976: [HUDI-106] Adding support for 
DynamicBloomFilter
URL: https://github.com/apache/incubator-hudi/pull/976#issuecomment-565845894
 
 
   @vinothchandar : the diff is ready. you are good to review. Sorry, forgot to 
ping you. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007869
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableMetaClient.java
 ##
 @@ -274,25 +286,35 @@ public synchronized HoodieArchivedTimeline 
getArchivedTimeline() {
*/
   public static HoodieTableMetaClient initTableType(Configuration hadoopConf, 
String basePath, String tableType,
   String tableName, String archiveLogFolder) throws IOException {
-HoodieTableType type = HoodieTableType.valueOf(tableType);
-Properties properties = new Properties();
-properties.put(HoodieTableConfig.HOODIE_TABLE_NAME_PROP_NAME, tableName);
-properties.put(HoodieTableConfig.HOODIE_TABLE_TYPE_PROP_NAME, type.name());
-properties.put(HoodieTableConfig.HOODIE_ARCHIVELOG_FOLDER_PROP_NAME, 
archiveLogFolder);
-return HoodieTableMetaClient.initDatasetAndGetMetaClient(hadoopConf, 
basePath, properties);
+return initTableType(hadoopConf, basePath, 
HoodieTableType.valueOf(tableType), tableName,
+archiveLogFolder, null, null);
   }
 
   /**
* Helper method to initialize a given path, as a given storage type and 
table name.
*/
   public static HoodieTableMetaClient initTableType(Configuration hadoopConf, 
String basePath,
   HoodieTableType tableType, String tableName, String payloadClassName) 
throws IOException {
+return initTableType(hadoopConf, basePath, tableType, tableName, null, 
payloadClassName, null);
+  }
+
+  public static HoodieTableMetaClient initTableType(Configuration hadoopConf, 
String basePath,
+  HoodieTableType tableType, String tableName, String archiveLogFolder, 
String payloadClassName,
+  Integer timelineLayoutVersion) throws IOException {
 Properties properties = new Properties();
 properties.setProperty(HoodieTableConfig.HOODIE_TABLE_NAME_PROP_NAME, 
tableName);
 properties.setProperty(HoodieTableConfig.HOODIE_TABLE_TYPE_PROP_NAME, 
tableType.name());
-if (tableType == HoodieTableType.MERGE_ON_READ) {
+if ((tableType == HoodieTableType.MERGE_ON_READ) && (payloadClassName != 
null)) {
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007840
 
 

 ##
 File path: hudi-common/src/main/avro/HoodieArchivedMetaEntry.avsc
 ##
 @@ -37,6 +37,7 @@
  "default": null
   },
   {
+ /** DEPRECATED **/
 
 Review comment:
   Yeah, This is was intended to be equivalent of HoodieCommitMetadata but 
since we are using HoodieCommitMetadata for storing results of compaction, this 
is never used. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007852
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/model/TimelineLayoutVersion.java
 ##
 @@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.model;
+
+import com.google.common.base.Preconditions;
+
+import java.io.Serializable;
+import java.util.Objects;
+
+/**
+ * Metadata Layout Version. Add new version when timeline format changes
+ */
+public class TimelineLayoutVersion implements Serializable, 
Comparable {
+
+  public static final Integer VERSION_0 = 0; // pre 0.5.1  version format
+  public static final Integer VERSION_1 = 1; // current version with no renames
+
+  public static final Integer CURR_VERSION = VERSION_1;
+  public static final TimelineLayoutVersion LATEST_TIMELINE_LAYOUT_VERSION = 
new TimelineLayoutVersion(CURR_VERSION);
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007673
 
 

 ##
 File path: hudi-common/src/main/avro/HoodieArchivedMetaEntry.avsc
 ##
 @@ -74,6 +75,27 @@
  "name":"version",
  "type":["int", "null"],
  "default": 1
+  },
+  {
+ "name":"hoodieCompactionPlan",
+ "type":[
+"null",
+"HoodieCompactionPlan"
+ ],
+ "default": null
+  },
+  {
+ "name":"hoodieCleanerPlan",
+ "type":[
+"null",
+"HoodieCleanerPlan"
+ ],
+ "default": null
+  },
+  {
+ "name":"actionState",
 
 Review comment:
   Yes, both actionType and instantTime are already available. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007647
 
 

 ##
 File path: 
hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
 ##
 @@ -733,6 +746,11 @@ public HoodieWriteConfig build() {
   setDefaultOnCondition(props, !isConsistencyGuardSet,
   ConsistencyGuardConfig.newBuilder().fromProperties(props).build());
 
+  String layoutVersion = 
props.getProperty(OVERRIDDEN_TIMELINE_LAYOUT_VERSION);
 
 Review comment:
   Done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007621
 
 

 ##
 File path: 
hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java
 ##
 @@ -48,6 +49,7 @@
 public class HoodieWriteConfig extends DefaultHoodieConfig {
 
   public static final String TABLE_NAME = "hoodie.table.name";
+  private static final String OVERRIDDEN_TIMELINE_LAYOUT_VERSION = 
"hoodie.timeline.layout.version";
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007569
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieInstant.java
 ##
 @@ -32,7 +35,24 @@
  *
  * @see HoodieTimeline
  */
-public class HoodieInstant implements Serializable {
+public class HoodieInstant implements Serializable, Comparable {
+
+  /**
+   * A COMPACTION action eventually becomes COMMIT when completed. So, when 
grouping instants
+   * for state transitions, this needs to be taken into account
+   */
+  private static final Map COMPARABLE_ACTIONS = new 
ImmutableMap.Builder()
+  .put(HoodieTimeline.COMPACTION_ACTION, 
HoodieTimeline.COMMIT_ACTION).build();
+
+  public static final Comparator ACTION_COMPARATOR =
+  Comparator.comparing(instant -> 
getCompatibleAction(instant.getAction()));
+
+  public static final Comparator COMPARATOR = 
Comparator.comparing(HoodieInstant::getTimestamp)
+  .thenComparing(ACTION_COMPARATOR).thenComparing(HoodieInstant::getState);
+
+  public static final String getCompatibleAction(String action) {
 
 Review comment:
   Picked Comparable. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007553
 
 

 ##
 File path: hudi-client/src/main/java/org/apache/hudi/HoodieWriteClient.java
 ##
 @@ -792,23 +798,25 @@ public void restoreToInstant(final String instantTime) 
throws HoodieRollbackExce
 .filter(instant -> 
HoodieActiveTimeline.GREATER.test(instant.getTimestamp(), instantTime))
 .collect(Collectors.toList());
 // Start a rollback instant for all commits to be rolled back
-String startRollbackInstant = startInstant();
+String startRollbackInstant = HoodieActiveTimeline.createNewCommitTime();
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007581
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
 ##
 @@ -61,11 +62,11 @@
 
   public static final SimpleDateFormat COMMIT_FORMATTER = new 
SimpleDateFormat("MMddHHmmss");
 
-  public static final Set VALID_EXTENSIONS_IN_ACTIVE_TIMELINE =
-  new HashSet<>(Arrays.asList(new String[] {COMMIT_EXTENSION, 
INFLIGHT_COMMIT_EXTENSION, DELTA_COMMIT_EXTENSION,
-  INFLIGHT_DELTA_COMMIT_EXTENSION, SAVEPOINT_EXTENSION, 
INFLIGHT_SAVEPOINT_EXTENSION, CLEAN_EXTENSION,
-  INFLIGHT_CLEAN_EXTENSION, REQUESTED_CLEAN_EXTENSION, 
INFLIGHT_COMPACTION_EXTENSION,
-  REQUESTED_COMPACTION_EXTENSION, INFLIGHT_RESTORE_EXTENSION, 
RESTORE_EXTENSION}));
+  public static final Set VALID_EXTENSIONS_IN_ACTIVE_TIMELINE = new 
HashSet<>(Arrays.asList(
+  new String[]{COMMIT_EXTENSION, INFLIGHT_COMMIT_EXTENSION, 
REQUESTED_COMMIT_EXTENTSION, DELTA_COMMIT_EXTENSION,
+  INFLIGHT_DELTA_COMMIT_EXTENSION, REQUESTED_DELTA_COMMIT_EXTENTSION, 
SAVEPOINT_EXTENSION,
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007546
 
 

 ##
 File path: 
hudi-cli/src/main/java/org/apache/hudi/cli/commands/DatasetsCommand.java
 ##
 @@ -85,6 +86,8 @@ public String createTable(
   @CliOption(key = {"tableName"}, mandatory = true, help = "Hoodie Table 
Name") final String name,
   @CliOption(key = {"tableType"}, unspecifiedDefaultValue = 
"COPY_ON_WRITE",
   help = "Hoodie Table Type. Must be one of : COPY_ON_WRITE or 
MERGE_ON_READ") final String tableTypeStr,
+  @CliOption(key = {"archiveLogFolder"}, help = "Folder Name for storing 
archived timeline") String archiveFolder,
+  @CliOption(key = {"layoutVersion"}, help = "Specific Layouyt Version to 
use") Integer layoutVersion,
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] bvaradar commented on a change in pull request #1009: [HUDI-308] Avoid Renames for tracking state transitions of all actions on dataset

[GitBox]

bvaradar commented on a change in pull request #1009:  [HUDI-308] Avoid Renames 
for tracking state transitions of all actions on dataset
URL: https://github.com/apache/incubator-hudi/pull/1009#discussion_r358007597
 
 

 ##
 File path: 
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieActiveTimeline.java
 ##
 @@ -61,11 +62,11 @@
 
   public static final SimpleDateFormat COMMIT_FORMATTER = new 
SimpleDateFormat("MMddHHmmss");
 
-  public static final Set VALID_EXTENSIONS_IN_ACTIVE_TIMELINE =
-  new HashSet<>(Arrays.asList(new String[] {COMMIT_EXTENSION, 
INFLIGHT_COMMIT_EXTENSION, DELTA_COMMIT_EXTENSION,
-  INFLIGHT_DELTA_COMMIT_EXTENSION, SAVEPOINT_EXTENSION, 
INFLIGHT_SAVEPOINT_EXTENSION, CLEAN_EXTENSION,
-  INFLIGHT_CLEAN_EXTENSION, REQUESTED_CLEAN_EXTENSION, 
INFLIGHT_COMPACTION_EXTENSION,
-  REQUESTED_COMPACTION_EXTENSION, INFLIGHT_RESTORE_EXTENSION, 
RESTORE_EXTENSION}));
+  public static final Set VALID_EXTENSIONS_IN_ACTIVE_TIMELINE = new 
HashSet<>(Arrays.asList(
+  new String[]{COMMIT_EXTENSION, INFLIGHT_COMMIT_EXTENSION, 
REQUESTED_COMMIT_EXTENTSION, DELTA_COMMIT_EXTENSION,
 
 Review comment:
   Done


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-hudi] nsivabalan commented on issue #976: [HUDI-106] Adding support for DynamicBloomFilter

[GitBox]

nsivabalan commented on issue #976: [HUDI-106] Adding support for 
DynamicBloomFilter
URL: https://github.com/apache/incubator-hudi/pull/976#issuecomment-565845894
 
 
   @vinothchandar : the diff is ready. you are good to review. Sorry, forgot 
you ping. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services