liyujiel opened a new pull request, #9562: URL: https://github.com/apache/pinot/pull/9562
Signed-off-by: Yujie Li <[email protected]> bugfix for docker and Kubernetes quick start. With the wrong path it will fail to start the load data job with the error: ``` k logs pinot-realtime-quickstart-load-data-into-kafka--1-x2klh Defaulted container "loading-json-data-to-kafka" out of: loading-json-data-to-kafka, loading-avro-data-to-kafka SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar) to method java.lang.Object.finalize() WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 2022/10/09 06:42:01.727 INFO [StreamAvroIntoKafkaCommand] [main] Streaming Avro file into Kafka topic flights-realtime with 1000 ms between messages java.io.FileNotFoundException: examples/stream/airlineStats/sample_data/airlineStats_data.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:219) at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) at org.apache.pinot.plugin.inputformat.avro.AvroUtils.getAvroReader(AvroUtils.java:220) at org.apache.pinot.tools.admin.command.StreamAvroIntoKafkaCommand.execute(StreamAvroIntoKafkaCommand.java:127) at org.apache.pinot.tools.Command.call(Command.java:33) at org.apache.pinot.tools.Command.call(Command.java:29) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:167) at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:198) java.lang.RuntimeException: java.io.FileNotFoundException: examples/stream/airlineStats/sample_data/airlineStats_data.avro (No such file or directory) at org.apache.pinot.tools.admin.command.StreamAvroIntoKafkaCommand.execute(StreamAvroIntoKafkaCommand.java:157) at org.apache.pinot.tools.Command.call(Command.java:33) at org.apache.pinot.tools.Command.call(Command.java:29) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at org.apache.pinot.tools.admin.PinotAdministrator.execute(PinotAdministrator.java:167) at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:198) Caused by: java.io.FileNotFoundException: examples/stream/airlineStats/sample_data/airlineStats_data.avro (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:219) at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) at org.apache.pinot.plugin.inputformat.avro.AvroUtils.getAvroReader(AvroUtils.java:220) at org.apache.pinot.tools.admin.command.StreamAvroIntoKafkaCommand.execute(StreamAvroIntoKafkaCommand.java:127) ``` I find the avro path by: ``` ~/pinot find . -name airlineStats_data.avro ./pinot/pinot-tools/src/main/resources/examples/stream/airlineStats/rawdata/airlineStats_data.avro ``` And rerun the data load job again, it can properly load data now! ``` stern pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh + pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh › loading-json-data-to-kafka + pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh › loading-avro-data-to-kafka pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Class path contains multiple SLF4J bindings. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Class path contains multiple SLF4J bindings. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-stream-ingestion/pinot-pulsar/pinot-pulsar-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-dropwizard/pinot-dropwizard-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-metrics/pinot-yammer/pinot-yammer-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-input-format/pinot-parquet/pinot-parquet-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-file-system/pinot-s3/pinot-s3-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Found binding in [jar:file:/opt/pinot/plugins/pinot-environment/pinot-azure/pinot-azure-0.12.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: An illegal reflective access operation has occurred pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar) to method java.lang.Object.finalize() pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka WARNING: All illegal access operations will be denied in a future release pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-json-data-to-kafka 2022/10/09 06:53:17.145 INFO [StreamAvroIntoKafkaCommand] [main] Streaming Avro file into Kafka topic flights-realtime with 1000 ms between messages pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: An illegal reflective access operation has occurred pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/opt/pinot/lib/pinot-all-0.12.0-SNAPSHOT-jar-with-dependencies.jar) to method java.lang.Object.finalize() pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka WARNING: All illegal access operations will be denied in a future release pinot-realtime-quickstart-load-data-into-kafka--1-wxrxh loading-avro-data-to-kafka 2022/10/09 06:53:21.349 INFO [StreamAvroIntoKafkaCommand] [main] Streaming Avro file into Kafka topic flights-realtime-avro with 1000 ms between messages ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
