Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory closed pull request #8042: NIFI-12382: Add DatabaseSchemaRegistry service URL: https://github.com/apache/nifi/pull/8042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on code in PR #8042:
URL: https://github.com/apache/nifi/pull/8042#discussion_r1415801708
##
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/src/main/java/org/apache/nifi/db/schemaregistry/DatabaseTableSchemaRegistry.java:
##
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.db.schemaregistry;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.dbcp.DBCPService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.SchemaIdentifier;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+@Tags({"schema", "registry", "database", "table"})
+@CapabilityDescription("Provides a service for generating a record schema from
a database table definition. The service is configured "
++ "to use a table name and a database connection fetches the table
metadata (i.e. table definition) such as column names, data types, "
++ "nullability, etc.")
+public class DatabaseTableSchemaRegistry extends AbstractControllerService
implements SchemaRegistry {
+
+private static final Set schemaFields =
EnumSet.of(SchemaField.SCHEMA_NAME);
+
+static final PropertyDescriptor DBCP_SERVICE = new
PropertyDescriptor.Builder()
+.name("Database Connection Pooling Service")
+.displayName("Database Connection Pooling Service")
+.description("The Controller Service that is used to obtain a
connection to the database for retrieving table information.")
+.required(true)
+.identifiesControllerService(DBCPService.class)
+.build();
+
+static final PropertyDescriptor CATALOG_NAME = new
PropertyDescriptor.Builder()
+.name("Catalog Name")
+.displayName("Catalog Name")
+.description("The name of the catalog used to locate the desired
table. This may not apply for the database that you are querying. In this case,
leave the field empty. Note that if the "
++ "property is set and the database is case-sensitive, the
catalog name must match the database's catalog name exactly.")
+.required(false)
+.expressionLanguageSupported(ExpressionLanguageScope.ENVIRONMENT)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+static final PropertyDescriptor SCHEMA_NAME = new
PropertyDescriptor.Builder()
+.name("Schema Name")
+.displayName("Schema Name")
+.description("The name of the schema that the table belongs to.
This may not apply for the database that you are updating. In this case, leave
the field empty. Note that if the "
++ "property is set and the database is case-sensitive, the
schema name must match the database's schema name exactly. Also notice that if
the same table name exists in multiple "
++ "schemas and Schema Name is not specified, the service
will find those tables and give an error if the different tables have the same
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on code in PR #8042: URL: https://github.com/apache/nifi/pull/8042#discussion_r1398201746 ## nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/pom.xml: ## @@ -0,0 +1,76 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-dbcp-service-bundle +2.0.0-SNAPSHOT + + +nifi-db-schema-registry-service +jar + + +org.apache.nifi +nifi-schema-registry-service-api + + +org.apache.nifi +nifi-dbcp-service-api + + +org.apache.nifi +nifi-dbcp-base +2.0.0-SNAPSHOT + Review Comment: Thanks for clarifying, defining the properties for the test class looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on code in PR #8042:
URL: https://github.com/apache/nifi/pull/8042#discussion_r1397676848
##
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/src/main/java/org/apache/nifi/db/schemaregistry/DatabaseSchemaRegistry.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.db.schemaregistry;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.dbcp.DBCPService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.SchemaIdentifier;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+@Tags({"schema", "registry", "database", "table"})
+@CapabilityDescription("Provides a service for generating a record schema from
a database table definition. The service is configured "
++ "to use a table name and a database connection fetches the table
metadata (i.e. table definition) such as column names, data types, "
++ "nullability, etc.")
+public class DatabaseSchemaRegistry extends AbstractControllerService
implements SchemaRegistry {
+
+private static final Set schemaFields =
EnumSet.of(SchemaField.SCHEMA_NAME);
+
+static final PropertyDescriptor DBCP_SERVICE = new
PropertyDescriptor.Builder()
+.name("dbcp-service")
Review Comment:
Yes, it is newer with part of the changes for migrating properties in NiFi
2.0. I know there have been past discussions about internationalization, but
unfortunately it appears there was never any significant progress in that
direction. For now, the best approach seems to be using the same name in both
places, and if there is ever momentum around internationalization, then we
could revisit a migration strategy. Unfortunately the disconnect between
property and display names right now makes it more difficult to translate from
a UI to programmatic approach, so keeping the same names will help under the
current circumstances.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
mattyb149 commented on code in PR #8042:
URL: https://github.com/apache/nifi/pull/8042#discussion_r1397661227
##
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/src/main/java/org/apache/nifi/db/schemaregistry/DatabaseSchemaRegistry.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.db.schemaregistry;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.dbcp.DBCPService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.SchemaIdentifier;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+@Tags({"schema", "registry", "database", "table"})
+@CapabilityDescription("Provides a service for generating a record schema from
a database table definition. The service is configured "
++ "to use a table name and a database connection fetches the table
metadata (i.e. table definition) such as column names, data types, "
++ "nullability, etc.")
+public class DatabaseSchemaRegistry extends AbstractControllerService
implements SchemaRegistry {
+
+private static final Set schemaFields =
EnumSet.of(SchemaField.SCHEMA_NAME);
+
+static final PropertyDescriptor DBCP_SERVICE = new
PropertyDescriptor.Builder()
+.name("dbcp-service")
Review Comment:
Is that the new convention? We used to do the opposite when requiring both
`name()` and `displayName()` to be set, `name` was a more machine-friendly name
for things like internationalization and `displayName` was the display name in
English.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
mattyb149 commented on code in PR #8042: URL: https://github.com/apache/nifi/pull/8042#discussion_r1397658749 ## nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/pom.xml: ## @@ -0,0 +1,76 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-dbcp-service-bundle +2.0.0-SNAPSHOT + + +nifi-db-schema-registry-service +jar + + +org.apache.nifi +nifi-schema-registry-service-api + + +org.apache.nifi +nifi-dbcp-service-api + + +org.apache.nifi +nifi-dbcp-base +2.0.0-SNAPSHOT + + +org.apache.nifi +nifi-kerberos-credentials-service-api +2.0.0-SNAPSHOT +provided + + +org.apache.nifi +nifi-kerberos-user-service-api + Review Comment: I got an error when running tests, but it might be because of including nifi-dbcp-base. If I can refactor out the common properties I probably don't need these either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
mattyb149 commented on code in PR #8042: URL: https://github.com/apache/nifi/pull/8042#discussion_r1397657327 ## nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/pom.xml: ## @@ -0,0 +1,76 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-dbcp-service-bundle +2.0.0-SNAPSHOT + + +nifi-db-schema-registry-service +jar + + +org.apache.nifi +nifi-schema-registry-service-api + + +org.apache.nifi +nifi-dbcp-service-api + + +org.apache.nifi +nifi-dbcp-base +2.0.0-SNAPSHOT + Review Comment: It's only for the common properties. If there's a better spot to move those to I can refactor it as part of this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on code in PR #8042:
URL: https://github.com/apache/nifi/pull/8042#discussion_r1397647623
##
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/src/main/java/org/apache/nifi/db/schemaregistry/DatabaseSchemaRegistry.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.db.schemaregistry;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.dbcp.DBCPService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.serialization.record.SchemaIdentifier;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.io.IOException;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+@Tags({"schema", "registry", "database", "table"})
+@CapabilityDescription("Provides a service for generating a record schema from
a database table definition. The service is configured "
++ "to use a table name and a database connection fetches the table
metadata (i.e. table definition) such as column names, data types, "
++ "nullability, etc.")
+public class DatabaseSchemaRegistry extends AbstractControllerService
implements SchemaRegistry {
+
+private static final Set schemaFields =
EnumSet.of(SchemaField.SCHEMA_NAME);
+
+static final PropertyDescriptor DBCP_SERVICE = new
PropertyDescriptor.Builder()
+.name("dbcp-service")
Review Comment:
As a new Service, this is a good opportunity to use the same value for the
name and display name.
```suggestion
.name("Database Connection Pooling Service")
```
##
nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-db-schema-registry-service/src/main/java/org/apache/nifi/db/schemaregistry/DatabaseSchemaRegistry.java:
##
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.db.schemaregistry;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.dbcp.DBCPService;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on PR #8042: URL: https://github.com/apache/nifi/pull/8042#issuecomment-1816579844 Thanks for the additional input @pvillard31, that is helpful, and makes more sense now. I realize that I was not evaluating all the implementation details. Now I see that the implementation uses the Database Metadata results, and that is what defines the information. Sorry for missing that detail earlier @mattyb149. With that in mind, I can now see how this makes more sense as a way to use database table metadata for the schema definition. Perhaps including `Metadata` in Service class name would also help describe how this works. With that background, are there any concerns about JDBC driver support across vendors? I would expect that more popular database vendors should work with this approach, but some may not, so that may be worth highlighting under additional details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
pvillard31 commented on PR #8042: URL: https://github.com/apache/nifi/pull/8042#issuecomment-1816560628 I'm jumping in the conversation here but I don't think this is a very specific use case. The idea is that, in many cases, data would be sent into a database by NiFi and some users may not want to deal with schemas in NiFi or introduce an external Schema Registry. The idea here is to be able to retrieve the schema of the destination table in the NiFi flow so that it can be leveraged instead of the Infer Schema option which comes with the limitations and risks that we know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
exceptionfactory commented on PR #8042: URL: https://github.com/apache/nifi/pull/8042#issuecomment-1816528314 Thanks for the reply @mattyb149. Renaming the class could be helpful. Another potential concern is using the schema name as the table name. Aside from schema naming conventions potentially conflicting with database table naming requirements, this would also require a table for every schema, which seems like it could be difficult to maintain. Although this could make sense in some specific environments, it seems like this might not be a good fit for inclusion as a component for general usage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
mattyb149 commented on PR #8042: URL: https://github.com/apache/nifi/pull/8042#issuecomment-1815409306 I'll add more documentation around the use case and what this service does, but I didn't see any use for a DB-backed registry that just stores schema text (such as a String column containing an Avro schema) since we have other implementations that basically do that. I can rename the service DatabaseTableSchemaRegistry if that helps make it clearer what this service is for, but the idea is to generate the schema from metadata rather than store and retrieve schemas. This can be used by things like ValidateRecord in order to ensure that records will successfully go into a database with a downstream PutDatabaseRecord for example. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] NIFI-12382: Add DatabaseSchemaRegistry service [nifi]
mattyb149 opened a new pull request, #8042: URL: https://github.com/apache/nifi/pull/8042 # Summary [NIFI-12382](https://issues.apache.org/jira/browse/NIFI-12382) This PR adds a DatabaseSchemaRegistry controller service which gets the metadata for a specified database table and creates a RecordSchema from it. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [x] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
