candlerb opened a new issue #5620: cpp/python API: please log to stderr instead of stdout by default URL: https://github.com/apache/pulsar/issues/5620 **Is your feature request related to a problem? Please describe.** The pulsar python client is a thin wrapper around the C++ API. When it runs, it generates debugging messages, e.g. ``` 2019-11-11 16:01:46.169 INFO ConnectionPool:72 | Created connection for pulsar://localhost:6650 2019-11-11 16:01:46.170 INFO ClientConnection:324 | [127.0.0.1:51970 -> 127.0.0.1:6650] Connected to broker 2019-11-11 16:01:46.177 INFO HandlerBase:52 | [persistent://public/default/syslog, reader-1d1d3c, 0] Getting connection from pool 2019-11-11 16:01:46.178 INFO ConnectionPool:72 | Created connection for pulsar://ldex-pulsar.int.example.net:6650 2019-11-11 16:01:46.179 INFO ClientConnection:326 | [127.0.0.1:51974 -> 127.0.0.1:6650] Connected to broker through proxy. Logical broker: pulsar://ldex-pulsar.int.example.net:6650 2019-11-11 16:01:46.183 INFO ConsumerImpl:170 | [persistent://public/default/syslog, reader-1d1d3c, 0] Created consumer on broker [127.0.0.1:51974 -> 127.0.0.1:6650] ``` However, these messages are generated on stdout, not stderr. As a result, this causes problems for any client program which wishes to use stdin and stdout for data communication. Example: an rsyslog `omprog` program receives messages on stdin and sends acknowledgements on stdout. The debug messages sent by the pulsar API confuse rsyslog and cause it to crash. **Describe the solution you'd like** If debug messages are to be enabled by default, I would like them to be written to stderr rather than stdout. That is a change of behaviour. If that's considered a bad thing at this stage, then it could be controlled by a flag. **Describe alternatives you've considered** There *is* an option `pulsar.Client(log_conf_file_path=...)` which suggests it can be [configured](https://logging.apache.org/log4cxx/latest_stable/usage.html#Configuration). However, it doesn't appear to do anything. I can set it to the name of a file, and strace does not show that file being opened. I can set it to a non-existent file, and no error occurs. ``` $ cat testconf.py import pulsar client = pulsar.Client('pulsar://localhost:6650', log_conf_file_path="/tmp/log4cpp.conf") msg_id = pulsar.MessageId.earliest reader = client.create_reader('my-topic', msg_id) reader.read_next(100) $ strace -f python3 testconf.py 2>strace.out ... $ grep log4cpp strace.out $ ``` Note: I am using `pulsar-client (2.4.1)` installed via pip3 under Ubuntu 18.04/python 3.6.8. **Additional context** I see that there are two logger implementations in lib/Log4cxxLogger.cc and lib/SimpleLoggerImpl.cc It looks like the SimpleLogger is default, in CMakeLists.txt: ``` option(USE_LOG4CXX "Build with Log4cxx support" OFF) ``` And as far as I can see, SimpleLoggerImpl is [hard-coded](https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/lib/SimpleLoggerImpl.cc#L62) to write to stdout: ``` void log(Level level, int line, const std::string &message) { std::stringstream ss; printTimestamp(ss); ss << " " << level << " " << _logger << ":" << line << " | " << message << "\n"; std::cout << ss.str(); std::cout.flush(); } ``` This makes it very hard to redirect the logging. It looks like I might have to fork a child, talk to it on fd 3, and copy data back and forth.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
