[GitHub] [incubator-pinot] fx19880617 opened a new pull request #6012: Configurable segment generation job parallelism for Hadoop and Spark

fx19880617 opened a new pull request #6012: URL: https://github.com/apache/incubator-pinot/pull/6012 ## Description Adding field 'segmentCreationJobParallelism' to allow users to set segment generation job parallelism. Default to the number of input files. This can avoid issue of

[incubator-pinot] branch configurable_spark_job_parallelism created (now 6e2cf59)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch configurable_spark_job_parallelism in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. at 6e2cf59 Adding field 'segmentCreationJobParallelism' to allow users to set

[incubator-pinot] 01/01: Adding field 'segmentCreationJobParallelism' to allow users to set segment generation job parallelism. Default to the number of input files.

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a commit to branch configurable_spark_job_parallelism in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git commit 6e2cf599bb8623acef91836979a6cef4ef68bc02 Author: Xiang Fu AuthorDate: Sat Sep 12

[GitHub] [incubator-pinot] jackjlli merged pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli merged pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #5991: [TE] Moved interfaces and datalayer to a new thirdeye-spi module.

suvodeep-pyne commented on pull request #5991: URL: https://github.com/apache/incubator-pinot/pull/5991#issuecomment-690852472 Thanks a lot @akshayrai ! This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-pinot] adriancole commented on pull request #5917: Starts Broker and Server in parallel when using ServiceManager

adriancole commented on pull request #5917: URL: https://github.com/apache/incubator-pinot/pull/5917#issuecomment-690859024 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations

fx19880617 removed a comment on issue #6003: URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467 @snleee @elonazoulay This is an automated message from the Apache Git Service. To respond

[incubator-pinot] branch master updated: Fix extract method in AvroRecordExtractor class (#6005)

This is an automated email from the ASF dual-hosted git repository. jlli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git The following commit(s) were added to refs/heads/master by this push: new cadd61c Fix extract method in

[GitHub] [incubator-pinot] jackjlli commented on pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli commented on pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#issuecomment-691313748 > Could you add a test such that it fails with the original code, and passes with the old code? Test added.

[GitHub] [incubator-pinot] fx19880617 commented on pull request #6002: Fixing S3PinotFS List API returned partial results

fx19880617 commented on pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#issuecomment-690912063 We should also review other PinotFs implementations to ensure the similar issues are not presenting there.

[GitHub] [incubator-pinot] kishoreg merged pull request #5991: [TE] Moved interfaces and datalayer to a new thirdeye-spi module.

kishoreg merged pull request #5991: URL: https://github.com/apache/incubator-pinot/pull/5991 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

Jackie-Jiang commented on a change in pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#discussion_r487331483 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -78,6 +78,9 @@

[GitHub] [incubator-pinot] fx19880617 merged pull request #6006: Fix/data view dev serve

fx19880617 merged pull request #6006: URL: https://github.com/apache/incubator-pinot/pull/6006 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] ChethanUK commented on pull request #6006: Fix/data view dev serve

ChethanUK commented on pull request #6006: URL: https://github.com/apache/incubator-pinot/pull/6006#issuecomment-691278842 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 merged pull request #6002: Fixing S3PinotFS List API returned partial results

fx19880617 merged pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6002: Fixing S3PinotFS List API returned partial results

fx19880617 commented on a change in pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#discussion_r486817834 ## File path: pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java ## @@ -374,33 +375,39

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli commented on a change in pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#discussion_r487224829 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java

[GitHub] [incubator-pinot] jackjlli commented on pull request #6009: Adjust schema validation logic in AvroIngestionSchemaValidator

jackjlli commented on pull request #6009: URL: https://github.com/apache/incubator-pinot/pull/6009#issuecomment-691397062 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations

fx19880617 commented on issue #6003: URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690914135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 commented on pull request #5917: Starts Broker and Server in parallel when using ServiceManager

fx19880617 commented on pull request #5917: URL: https://github.com/apache/incubator-pinot/pull/5917#issuecomment-690867454 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

codecov-commenter commented on pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#issuecomment-691309424 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #6002: Fixing S3PinotFS List API returned partial results

kishoreg commented on a change in pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#discussion_r486810483 ## File path: pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java ## @@ -374,33 +375,39 @@

[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #6001: [TE] entity anomaly logging for ad-hoc debugging

jihaozh commented on a change in pull request #6001: URL: https://github.com/apache/incubator-pinot/pull/6001#discussion_r487263049 ## File path: thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/tools/RunAdhocDatabaseQueriesTool.java ## @@ -723,9 +741,65 @@

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

yupeng9 commented on a change in pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#discussion_r487290286 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -78,6 +78,9 @@ public

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6005: Fix extract method in AvroRecordExtractor class

Jackie-Jiang commented on a change in pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#discussion_r487213705 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java

[GitHub] [incubator-pinot] jackjlli commented on pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli commented on pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#issuecomment-691313748 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations

fx19880617 removed a comment on issue #6003: URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [incubator-pinot] kishoreg merged pull request #5991: [TE] Moved interfaces and datalayer to a new thirdeye-spi module.

kishoreg merged pull request #5991: URL: https://github.com/apache/incubator-pinot/pull/5991 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

Jackie-Jiang commented on a change in pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#discussion_r487331483 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -78,6 +78,9 @@

[GitHub] [incubator-pinot] fx19880617 merged pull request #6006: Fix/data view dev serve

fx19880617 merged pull request #6006: URL: https://github.com/apache/incubator-pinot/pull/6006 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] suvodeep-pyne commented on pull request #5991: [TE] Moved interfaces and datalayer to a new thirdeye-spi module.

suvodeep-pyne commented on pull request #5991: URL: https://github.com/apache/incubator-pinot/pull/5991#issuecomment-690852472 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [incubator-pinot] adriancole commented on pull request #5917: Starts Broker and Server in parallel when using ServiceManager

adriancole commented on pull request #5917: URL: https://github.com/apache/incubator-pinot/pull/5917#issuecomment-690859024 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6002: Fixing S3PinotFS List API returned partial results

fx19880617 commented on a change in pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#discussion_r486817834 ## File path: pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java ## @@ -374,33 +375,39

[GitHub] [incubator-pinot] jackjlli commented on a change in pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli commented on a change in pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#discussion_r487224829 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java

[GitHub] [incubator-pinot] jackjlli commented on pull request #6009: Adjust schema validation logic in AvroIngestionSchemaValidator

jackjlli commented on pull request #6009: URL: https://github.com/apache/incubator-pinot/pull/6009#issuecomment-691397062 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 commented on pull request #5917: Starts Broker and Server in parallel when using ServiceManager

fx19880617 commented on pull request #5917: URL: https://github.com/apache/incubator-pinot/pull/5917#issuecomment-690867454 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] fx19880617 commented on issue #6003: Review PinotFS ListFile Implementations

fx19880617 commented on issue #6003: URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690914135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [incubator-pinot] codecov-commenter commented on pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

codecov-commenter commented on pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#issuecomment-691309424 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [incubator-pinot] kishoreg commented on a change in pull request #6002: Fixing S3PinotFS List API returned partial results

kishoreg commented on a change in pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#discussion_r486810483 ## File path: pinot-plugins/pinot-file-system/pinot-s3/src/main/java/org/apache/pinot/plugin/filesystem/S3PinotFS.java ## @@ -374,33 +375,39 @@

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6005: Fix extract method in AvroRecordExtractor class

Jackie-Jiang commented on a change in pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#discussion_r487213705 ## File path: pinot-plugins/pinot-input-format/pinot-avro-base/src/main/java/org/apache/pinot/plugin/inputformat/avro/AvroRecordExtractor.java

[GitHub] [incubator-pinot] jackjlli commented on pull request #6005: Fix extract method in AvroRecordExtractor class

jackjlli commented on pull request #6005: URL: https://github.com/apache/incubator-pinot/pull/6005#issuecomment-691313748 > Could you add a test such that it fails with the original code, and passes with the old code? Test added.

[GitHub] [incubator-pinot] fx19880617 removed a comment on issue #6003: Review PinotFS ListFile Implementations

fx19880617 removed a comment on issue #6003: URL: https://github.com/apache/incubator-pinot/issues/6003#issuecomment-690920467 @snleee @elonazoulay This is an automated message from the Apache Git Service. To respond

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

yupeng9 commented on a change in pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#discussion_r487290286 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -78,6 +78,9 @@ public

[GitHub] [incubator-pinot] fx19880617 commented on pull request #6002: Fixing S3PinotFS List API returned partial results

fx19880617 commented on pull request #6002: URL: https://github.com/apache/incubator-pinot/pull/6002#issuecomment-690912063 We should also review other PinotFs implementations to ensure the similar issues are not presenting there.

[GitHub] [incubator-pinot] kishoreg merged pull request #5991: [TE] Moved interfaces and datalayer to a new thirdeye-spi module.

kishoreg merged pull request #5991: URL: https://github.com/apache/incubator-pinot/pull/5991 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [incubator-pinot] jihaozh commented on a change in pull request #6001: [TE] entity anomaly logging for ad-hoc debugging

jihaozh commented on a change in pull request #6001: URL: https://github.com/apache/incubator-pinot/pull/6001#discussion_r487263049 ## File path: thirdeye/thirdeye-pinot/src/test/java/org/apache/pinot/thirdeye/tools/RunAdhocDatabaseQueriesTool.java ## @@ -723,9 +741,65 @@

[GitHub] [incubator-pinot] fx19880617 merged pull request #6006: Fix/data view dev serve

fx19880617 merged pull request #6006: URL: https://github.com/apache/incubator-pinot/pull/6006 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6008: Add a length limit of 512 to the properties stored in the segment metadata

Jackie-Jiang commented on a change in pull request #6008: URL: https://github.com/apache/incubator-pinot/pull/6008#discussion_r487331483 ## File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java ## @@ -78,6 +78,9 @@

[incubator-pinot] branch update_pinot_helm_for_custom_config_file updated (c3d3d20 -> 33eaf2b)

This is an automated email from the ASF dual-hosted git repository. xiangfu pushed a change to branch update_pinot_helm_for_custom_config_file in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git. discard c3d3d20 Update pinot helm to adding custom configs and update the jvm