the code is in core.clj in mk-include-sys-fn which calls system-id?(common.clj). I think UI code is fine but nimbus won't accept a topology which has stream-id with "__" at the beginning. We can probably add a exception in nimbus if a stream starts with "__" and its in storm.user.system.streams. Can you please file a jira to track this. Thanks.
-Harsha On Mon, Jul 21, 2014, at 08:59 AM, Julien Nioche wrote: Yes, that's also the conclusion I came to. I could not find where in the UI code is the call to Utils.isSystemId(String). One option would be to be able to define in the configuration a list of streams to treat as system. Does the UI code access the configuration files at all? Thanks Harsha On 21 July 2014 16:43, Harsha <[1][email protected]> wrote: thats caused by validate-ids! function which checks if the users stream id is system id and throws that exception. So looks like "__" reserved for system streams only not allowed for users. On Mon, Jul 21, 2014, at 08:30 AM, Julien Nioche wrote: Hi Harsha Am getting : 5935 [main] WARN backtype.storm.daemon.nimbus - Topology submission exception. (topology name='QueuePopulator') #<InvalidTopologyException InvalidTopologyException(msg:__log is not a valid stream id)> 5941 [main] ERROR org.apache.zookeeper.server.NIOServerCnxnFactory - Thread Thread[main,5,main] died backtype.storm.generated.InvalidTopologyException: null at backtype.storm.daemon.common$validate_ids_BANG_.invoke(common.c lj:126) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] at backtype.storm.daemon.common$validate_basic_BANG_.invoke(common .clj:142) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] at backtype.storm.daemon.common$system_topology_BANG_.invoke(commo n.clj:297) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] Thanks Julien On 21 July 2014 16:22, Harsha <[2][email protected]> wrote: Hi Julien, UI code calls Utils.isSystemId(String) which checks if the stream id starts with "__". What error are you seeing when you renamed into "__log". -Harsha On Mon, Jul 21, 2014, at 03:45 AM, Julien Nioche wrote: Hi, I have a custom stream for handling logs (called '_log') and send them to ElasticSearch for indexing. The log tuples are generated by my spouts and bolts. My pipeline also uses the default stream for the normal processing of tuples from RabbitMQ. Everything works fine but I would like to be able to treat this _log stream as one of the system ones (e.g. __metrics) and be able to hide them from the stats. The summary of Emitted / Transferred currently takes these log events into account which is not very useful. I tried renaming the stream into '__log' but this resulted in an error when trying to start the topoloy. Any idea of how I could do that? Thanks Julien -- [logo.gif] Open Source Solutions for Text Engineering [3]http://digitalpebble.blogspot.com/ [4]http://www.digitalpebble.com [5]http://twitter.com/digitalpebble -- [logo.gif] Open Source Solutions for Text Engineering [6]http://digitalpebble.blogspot.com/ [7]http://www.digitalpebble.com [8]http://twitter.com/digitalpebble -- [logo.gif] Open Source Solutions for Text Engineering [9]http://digitalpebble.blogspot.com/ [10]http://www.digitalpebble.com [11]http://twitter.com/digitalpebble References 1. mailto:[email protected] 2. mailto:[email protected] 3. http://digitalpebble.blogspot.com/ 4. http://www.digitalpebble.com/ 5. http://twitter.com/digitalpebble 6. http://digitalpebble.blogspot.com/ 7. http://www.digitalpebble.com/ 8. http://twitter.com/digitalpebble 9. http://digitalpebble.blogspot.com/ 10. http://www.digitalpebble.com/ 11. http://twitter.com/digitalpebble
