[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363966330 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/test/resources/server.properties ## @@ -21,25 +22,27 @@ broker.id=0 # Socket Server Settings # -# The port the socket server listens on +# The address the socket server listens on. It will get the value returned from +# java.net.InetAddress.getCanonicalHostName() if not configured. +# FORMAT: +# listeners = listener_name://host_name:port +# EXAMPLE: +# listeners = PLAINTEXT://your.host.name:9092 +#listeners=PLAINTEXT://:9092 port=9092 -# Hostname the broker will bind to. If not set, the server will bind to all interfaces -#host.name=localhost - -# Hostname the broker will advertise to producers and consumers. If not set, it uses the -# value for "host.name" if configured. Otherwise, it will use the value returned from -# java.net.InetAddress.getCanonicalHostName(). -#advertised.host.name= +# Hostname and port the broker will advertise to producers and consumers. If not set, Review comment: See my comment about Kafka `server.properties` above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363966387 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/test/resources/server.properties ## @@ -21,25 +22,27 @@ broker.id=0 # Socket Server Settings # -# The port the socket server listens on +# The address the socket server listens on. It will get the value returned from +# java.net.InetAddress.getCanonicalHostName() if not configured. +# FORMAT: +# listeners = listener_name://host_name:port +# EXAMPLE: +# listeners = PLAINTEXT://your.host.name:9092 +#listeners=PLAINTEXT://:9092 port=9092 -# Hostname the broker will bind to. If not set, the server will bind to all interfaces -#host.name=localhost - -# Hostname the broker will advertise to producers and consumers. If not set, it uses the -# value for "host.name" if configured. Otherwise, it will use the value returned from -# java.net.InetAddress.getCanonicalHostName(). -#advertised.host.name= +# Hostname and port the broker will advertise to producers and consumers. If not set, +# it uses the value for "listeners" if configured. Otherwise, it will use the value +# returned from java.net.InetAddress.getCanonicalHostName(). +#advertised.listeners=PLAINTEXT://your.host.name:9092 -# The port to publish to ZooKeeper for clients to use. If this is not set, -# it will publish the same port that the broker binds to. -#advertised.port= +# Maps listener names to security protocols, the default is for them to be the same. See the config documentation for more details +#listener.security.protocol.map=PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL -# The number of threads handling network requests +# The number of threads that the server uses for receiving requests from the network and sending responses to the network Review comment: See my comment about Kafka `server.properties` above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363966250 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/test/resources/server.properties ## @@ -119,3 +125,13 @@ zookeeper.connect=localhost:2181 # Timeout in ms for connecting to zookeeper zookeeper.connection.timeout.ms=6000 + + +# Group Coordinator Settings # + +# The following configuration specifies the time, in milliseconds, that the GroupCoordinator will delay the initial consumer rebalance. +# The rebalance will be further delayed by the value of group.initial.rebalance.delay.ms as new members join the group, up to a maximum of max.poll.interval.ms. Review comment: See my comment about Kafka `server.properties` above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363966173 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/test/resources/server.properties ## @@ -66,14 +69,21 @@ num.partitions=1 # This value is recommended to be increased for installations with data dirs located in RAID array. num.recovery.threads.per.data.dir=1 +# Internal Topic Settings # +# The replication factor for the group metadata internal topics "__consumer_offsets" and "__transaction_state" +# For anything other than development testing, a value greater than 1 is recommended for to ensure availability such as 3. Review comment: See my comment about Kafka `server.properties` above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363965363 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/test/resources/server.properties ## @@ -21,25 +22,27 @@ broker.id=0 # Socket Server Settings # -# The port the socket server listens on +# The address the socket server listens on. It will get the value returned from Review comment: The `server.properties` file is an original Kafka 2.0 server config file with minimal changes. I would not edit the comments in it. I also removed the port=9092 property which was not necessary (and deprecated). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0
turcsanyip commented on a change in pull request #3944: NIFI-6939: Upgrade Atlas client dependency to 2.0.0 URL: https://github.com/apache/nifi/pull/3944#discussion_r363963580 ## File path: nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/main/java/org/apache/nifi/atlas/hook/NotificationSender.java ## @@ -290,17 +293,17 @@ void send(final List messages, final C // org.json4s.package$MappingException: Can't find ScalaSig for class org.apache.atlas.typesystem.Referenceable flowPathRef.set(ATTR_INPUTS, new ArrayList<>(distinctInputs.values())); flowPathRef.set(ATTR_OUTPUTS, new ArrayList<>(distinctOutputs.values())); -return new HookNotification.EntityPartialUpdateRequest(NIFI_USER, TYPE_NIFI_FLOW_PATH, +return new HookNotificationV1.EntityPartialUpdateRequest(NIFI_USER, TYPE_NIFI_FLOW_PATH, ATTR_QUALIFIED_NAME, flowPathQualifiedName, flowPathRef); }) .filter(Objects::nonNull) .collect(Collectors.toList()); metrics.uniquePartialNiFiFlowPathUpdates = deduplicatedMessages.size(); -notifier.accept(deduplicatedMessages); +notifier.accept(deduplicatedMessages, null); Review comment: Good point. I retested the reporting task with kerberized Atlas servers (version 1.x and 2.0) and it worked fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
