exceptionfactory commented on code in PR #7544:
URL: https://github.com/apache/nifi/pull/7544#discussion_r1412711140
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -1276,8 +1303,10 @@ SqlAndIncludedColumns generateUpdate(final RecordSchema
recordSchema, final Stri
String fieldName = field.getFieldName();
boolean firstUpdateKey = true;
- final String normalizedColName =
ColumnDescription.normalizeColumnName(fieldName, settings.translateFieldNames);
- final ColumnDescription desc =
tableSchema.getColumns().get(ColumnDescription.normalizeColumnName(fieldName,
settings.translateFieldNames));
+ final String normalizedColName = getNormalizedName(fieldName,
+ settings.translateFieldNames,
settings.translationStrategy, settings.translationRegex);
+ final ColumnDescription desc =
tableSchema.getColumns().get(getNormalizedName(fieldName,
+ settings.translateFieldNames,
settings.translationStrategy, settings.translationRegex));
Review Comment:
```suggestion
final ColumnDescription desc =
tableSchema.getColumns().get(normalizedColName);
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -1577,6 +1610,8 @@ static class DMLSettings {
DMLSettings(ProcessContext context) {
translateFieldNames =
context.getProperty(TRANSLATE_FIELD_NAMES).asBoolean();
+ translationStrategy =
TranslationStrategy.valueOf(context.getProperty(TRANSLATION_STRATEGY).getValue());
+ translationRegex =
context.getProperty(TRANSLATION_REGEX).getValue();
Review Comment:
This may need to be null-checked, but should be declared with
`Pattern.compile()`
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/TranslationStrategy.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db;
+
+import org.apache.nifi.components.DescribedValue;
+
+/**
+ * Enumeration of supported Database column name Translation Strategy
+ */
+public enum TranslationStrategy implements DescribedValue {
+ REMOVE_UNDERSCORE("REMOVE_UNDERSCORE","Remove Underscore","Underscore(_)
will be removed from column name with empty string Ex. Pics_1_11 become
PICS111"),
+ REMOVE_SPACE("REMOVE_SPACE","Remove Space","Spaces will be removed from
column name with empty string Ex. 'User Name' become 'USERNAME'"),
+ REMOVE_ALL_SPECIAL_CHAR("REMOVE_ALL_SPECIAL_CHAR","Remove All Special
Character","Remove All Special Character"),
+ REGEX("REGEX","Regular Expression","Remove character matched Regular
Expression from column name");
+ private final String value;
+
+ private final String displayName;
+
+ private final String description;
+ TranslationStrategy(final String value, final String displayName, final
String description) {
+ this.value = value;
+ this.displayName = displayName;
+ this.description = description;
+ }
+
+ @Override
+ public String getValue() {
+ return value;
Review Comment:
```suggestion
return name();
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -246,6 +249,25 @@ public class PutDatabaseRecord extends AbstractProcessor {
.defaultValue("true")
.build();
+ public static final PropertyDescriptor TRANSLATION_STRATEGY = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("translation-strategy")
+ .description("The strategy used to normalize table column name")
+ .allowableValues(TranslationStrategy.class)
+ .defaultValue(TranslationStrategy.REMOVE_UNDERSCORE.getValue())
+ .dependsOn(TRANSLATE_FIELD_NAMES,
TRANSLATE_FIELD_NAMES.getDefaultValue())
+ .build();
+
+ public static final PropertyDescriptor TRANSLATION_REGEX = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("translation-regex")
+ .displayName("Translation Regular Expression")
Review Comment:
```suggestion
.name("Column Name Translation Pattern")
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -246,6 +249,25 @@ public class PutDatabaseRecord extends AbstractProcessor {
.defaultValue("true")
.build();
+ public static final PropertyDescriptor TRANSLATION_STRATEGY = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("translation-strategy")
+ .description("The strategy used to normalize table column name")
+ .allowableValues(TranslationStrategy.class)
+ .defaultValue(TranslationStrategy.REMOVE_UNDERSCORE.getValue())
+ .dependsOn(TRANSLATE_FIELD_NAMES,
TRANSLATE_FIELD_NAMES.getDefaultValue())
+ .build();
+
+ public static final PropertyDescriptor TRANSLATION_REGEX = new
PropertyDescriptor.Builder()
Review Comment:
```suggestion
public static final PropertyDescriptor TRANSLATION_PATTERN = new
PropertyDescriptor.Builder()
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -1231,13 +1257,14 @@ SqlAndIncludedColumns generateUpdate(final RecordSchema
recordSchema, final Stri
int fieldCount = fieldNames.size();
AtomicInteger fieldsFound = new AtomicInteger(0);
-
for (int i = 0; i < fieldCount; i++) {
RecordField field = recordSchema.getField(i);
String fieldName = field.getFieldName();
- final String normalizedColName =
ColumnDescription.normalizeColumnName(fieldName, settings.translateFieldNames);
- final ColumnDescription desc =
tableSchema.getColumns().get(ColumnDescription.normalizeColumnName(fieldName,
settings.translateFieldNames));
+ final String normalizedColName = getNormalizedName(fieldName,
settings.translateFieldNames,
+ settings.translationStrategy,
settings.translationRegex);
+ final ColumnDescription desc =
tableSchema.getColumns().get(getNormalizedName(fieldName,
+ settings.translateFieldNames,
settings.translationStrategy, settings.translationRegex));
Review Comment:
This appears to be an unnecessary call to `getNormalizedName()` when
`normalizedColName` can be used.
```suggestion
final ColumnDescription desc =
tableSchema.getColumns().get(normalizedColName);
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/TranslationStrategy.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db;
+
+import org.apache.nifi.components.DescribedValue;
+
+/**
+ * Enumeration of supported Database column name Translation Strategy
+ */
+public enum TranslationStrategy implements DescribedValue {
+ REMOVE_UNDERSCORE("REMOVE_UNDERSCORE","Remove Underscore","Underscore(_)
will be removed from column name with empty string Ex. Pics_1_11 become
PICS111"),
+ REMOVE_SPACE("REMOVE_SPACE","Remove Space","Spaces will be removed from
column name with empty string Ex. 'User Name' become 'USERNAME'"),
+ REMOVE_ALL_SPECIAL_CHAR("REMOVE_ALL_SPECIAL_CHAR","Remove All Special
Character","Remove All Special Character"),
+ REGEX("REGEX","Regular Expression","Remove character matched Regular
Expression from column name");
+ private final String value;
+
+ private final String displayName;
+
+ private final String description;
+ TranslationStrategy(final String value, final String displayName, final
String description) {
+ this.value = value;
Review Comment:
The `value` can be removed and the `name()` of the enum can be used in
`getValue()`.
```suggestion
TranslationStrategy(final String displayName, final String description) {
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/ColumnNameNormalizer.java:
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard.db;
+public interface ColumnNameNormalizer {
+
+
+ /**
+ * Normalizes the given column name based on the specified strategy.
+ *
+ * @param colName The column name to be normalized.
+ * @param isTranslationEnabled Boolean value to denote normalization is
enabled
+ * @param strategy The TranslationStrategy for normalizing column name
+ * @param translationRegex Regex For translation
+ * @return The normalized column name as a String.
+ */
+ default String getNormalizedName(final String colName,boolean
isTranslationEnabled, TranslationStrategy strategy, String translationRegex) {
Review Comment:
```suggestion
default String getNormalizedName(String colName, boolean
translationEnabled, TranslationStrategy strategy, Pattern translationPattern) {
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UpdateDatabaseTable.java:
##########
@@ -177,7 +179,25 @@ public class UpdateDatabaseTable extends AbstractProcessor
{
.allowableValues("true", "false")
.defaultValue("true")
.build();
+ public static final PropertyDescriptor TRANSLATION_STRATEGY = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("Translation Strategy")
+ .description("The strategy used to normalize table column name")
+ .allowableValues(TranslationStrategy.class)
+ .defaultValue(TranslationStrategy.REMOVE_UNDERSCORE.getValue())
+ .dependsOn(TRANSLATE_FIELD_NAMES,
TRANSLATE_FIELD_NAMES.getDefaultValue())
+ .build();
+ public static final PropertyDescriptor TRANSLATION_REGEX = new
PropertyDescriptor.Builder()
+ .name("translation-regex")
+ .displayName("Translation Regular Expression")
Review Comment:
```suggestion
.name("Column Name Translation Pattern")
.displayName("Column Name Translation Pattern")
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -246,6 +249,25 @@ public class PutDatabaseRecord extends AbstractProcessor {
.defaultValue("true")
.build();
+ public static final PropertyDescriptor TRANSLATION_STRATEGY = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("translation-strategy")
Review Comment:
```suggestion
.name("Column Name Translation Strategy")
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/ColumnNameNormalizer.java:
##########
@@ -0,0 +1,35 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard.db;
+public interface ColumnNameNormalizer {
Review Comment:
This should be spaced.
```suggestion
public interface ColumnNameNormalizer {
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UpdateDatabaseTable.java:
##########
@@ -177,7 +179,25 @@ public class UpdateDatabaseTable extends AbstractProcessor
{
.allowableValues("true", "false")
.defaultValue("true")
.build();
+ public static final PropertyDescriptor TRANSLATION_STRATEGY = new
PropertyDescriptor.Builder()
+ .required(true)
+ .name("Translation Strategy")
Review Comment:
```suggestion
.name("Column Name Translation Strategy")
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/TranslationStrategy.java:
##########
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard.db;
+
+import org.apache.nifi.components.DescribedValue;
+
+/**
+ * Enumeration of supported Database column name Translation Strategy
+ */
+public enum TranslationStrategy implements DescribedValue {
+ REMOVE_UNDERSCORE("REMOVE_UNDERSCORE","Remove Underscore","Underscore(_)
will be removed from column name with empty string Ex. Pics_1_11 become
PICS111"),
+ REMOVE_SPACE("REMOVE_SPACE","Remove Space","Spaces will be removed from
column name with empty string Ex. 'User Name' become 'USERNAME'"),
+ REMOVE_ALL_SPECIAL_CHAR("REMOVE_ALL_SPECIAL_CHAR","Remove All Special
Character","Remove All Special Character"),
+ REGEX("REGEX","Regular Expression","Remove character matched Regular
Expression from column name");
Review Comment:
```suggestion
PATTERN("Regular Expression", "Remove character matched Regular
Expression from column name");
```
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UpdateDatabaseTable.java:
##########
@@ -374,6 +396,9 @@ public void onTrigger(final ProcessContext context, final
ProcessSession session
final boolean createIfNotExists =
context.getProperty(CREATE_TABLE).getValue().equals(CREATE_IF_NOT_EXISTS.getValue());
final boolean updateFieldNames =
context.getProperty(UPDATE_FIELD_NAMES).asBoolean();
final boolean translateFieldNames =
context.getProperty(TRANSLATE_FIELD_NAMES).asBoolean();
+ final TranslationStrategy translationStrategy =
TranslationStrategy.valueOf(context.getProperty(TRANSLATION_STRATEGY).getValue());
+ final String translationRegex =
context.getProperty(TRANSLATION_REGEX).getValue();
Review Comment:
This should be null-checked and parsed with `Pattern.compile()`
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutDatabaseRecord.java:
##########
@@ -1563,6 +1594,8 @@ public String apply(final Record record) {
static class DMLSettings {
private final boolean translateFieldNames;
+ private final TranslationStrategy translationStrategy;
+ private final String translationRegex;
Review Comment:
This should be declared as a `Pattern` for more efficient repeated
processing.
##########
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/db/ColumnNameNormalizerUtility.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.standard.db;
+
+
+import java.util.Objects;
+
+/**
+ * ColumnNameNormalizerUtility is a utility class that helps to normalize
column names. It provides various strategies to
+ * modify column names based on the TranslationStrategy enum. Column names can
be normalized by removing underscores,
+ * spaces, all special characters, or by using a custom regular expression
defined by the user.
+ */
+public class ColumnNameNormalizerUtility {
+ // Regular expression to remove all special characters from a string.
+ private static final String REMOVE_ALL_SPECIAL_CHAR_REGEX = "[^a-zA-Z0-9]";
Review Comment:
`Pattern.compile()` should be used to preset the parsed pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
