kad3nce opened a new issue #6992: Is this supervisor spec well-formed? URL: https://github.com/apache/incubator-druid/issues/6992 I am trying to perform a realtime ingestion task from Kafka using the Kafka indexing service plugin. I am loading the requisite plugins for my scenario (using Divolte with an Avro schema): ```druid.extensions.loadList=["druid-kafka-indexing-service", "druid-avro-extensions", "mysql-metadata-storage"]``` But submitting the supervisor spec does not seem to be accomplishing the task of creating the datasource in Druid. ```sh $ curl -XPOST -H'Content-Type: application/json' -d @supervisor-spec.json http://localhost:8090/druid/indexer/v1/supervisor {"id":"livestream"} ```` After submitting this supervisor spec, I can see the supervisor in the Overlord console:  But the datasource is listed as 'undefined' in the **indexing** tab of the Druid Cluster monitor:  And is not listed at all in the **datasources** tab:  I was under the impression that submitting a supervisor spec was supposed to result in that supervisor spec's datasource being automatically created by Druid. Is that not the case? Is there something wrong with my spec? ```json { "type": "kafka", "dataSchema": { "dataSource": "livestream", "parser": { "type": "avro_stream", "avroBytesDecoder": { "type": "schema_inline", "subjectAndIdConverter": { "type": "avro_1124", "topic": "livestream" }, "schema": { "namespace": "org.apache.druid.data", "name": "User", "type": "record", "fields": [{ "name": "detectedDuplicate", "type": "boolean" }, { "name": "detectedCorruption", "type": "boolean" }, { "name": "firstInSession", "type": "boolean" }, { "name": "timestamp", "type": "long" }, { "name": "remoteHost", "type": "string" }, { "name": "referer", "type": ["null", "string"], "default": null }, { "name": "location", "type": ["null", "string"], "default": null }, { "name": "viewportPixelWidth", "type": ["null", "int"], "default": null }, { "name": "viewportPixelHeight", "type": ["null", "int"], "default": null }, { "name": "screenPixelWidth", "type": ["null", "int"], "default": null }, { "name": "screenPixelHeight", "type": ["null", "int"], "default": null }, { "name": "partyId", "type": ["null", "string"], "default": null }, { "name": "sessionId", "type": ["null", "string"], "default": null }, { "name": "pageViewId", "type": ["null", "string"], "default": null }, { "name": "eventType", "type": "string", "default": "unknown" }, { "name": "userAgentString", "type": ["null", "string"], "default": null }, { "name": "userAgentName", "type": ["null", "string"], "default": null }, { "name": "userAgentFamily", "type": ["null", "string"], "default": null }, { "name": "userAgentVendor", "type": ["null", "string"], "default": null }, { "name": "userAgentType", "type": ["null", "string"], "default": null }, { "name": "userAgentVersion", "type": ["null", "string"], "default": null }, { "name": "userAgentDeviceCategory", "type": ["null", "string"], "default": null }, { "name": "userAgentOsFamily", "type": ["null", "string"], "default": null }, { "name": "userAgentOsVersion", "type": ["null", "string"], "default": null }, { "name": "userAgentOsVendor", "type": ["null", "string"], "default": null }, { "name": "technology", "type": ["null", "string"], "default": null } ] } }, "parseSpec": { "format": "timeAndDims", "timestampSpec": { "column": "timestamp", "format": "auto" }, "dimensionsSpec": { "dimensions": [ "detectedDuplicate", "detectedCorruption", "firstInSession", "remoteHost", "referer", "location", "viewportPixelWidth", "viewportPixelHeight", "screenPixelWidth", "screenPixelHeight", "partyId", "sessionId", "pageViewId", "eventType", "userAgentString", "userAgentName", "userAgentFamily", "userAgentVendor", "userAgentType", "userAgentVersion", "userAgentDeviceCategory", "userAgentOsFamily", "userAgentOsVersion", "userAgentOsVendor", "technology" ] } } }, "metricsSpec": [{ "name": "count", "type": "count" }], "granularitySpec": { "type": "uniform", "segmentGranularity": "HOUR", "queryGranularity": "NONE" } }, "tuningConfig": { "type": "kafka", "maxRowsPerSegment": 5000000, "logParseExceptions": true }, "ioConfig": { "topic": "livestream", "consumerProperties": { "bootstrap.servers": "10.0.1.32:9092" }, "taskCount": 1, "replicas": 1, "taskDuration": "PT5S" } } ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
