> On Nov. 1, 2014, 2:34 a.m., Jonathan Hurley wrote: > > ambari-agent/src/main/python/ambari_agent/loghandler/hdfsapi.py, line 31 > > <https://reviews.apache.org/r/27396/diff/2/?file=746522#file746522line31> > > > > I'm not sure this command will work for all deployments. In some cases, > > the "hdfs" user is the only one that can run these commands; and in > > security-enabled environments, that means using keytabs. > > > > Can you verify that this command will work in the above scenarios? > > > > Same goes for other hdfs commands. > > Cabir Zounaidou wrote: > Good point. Verified that it is an issue for 'ambari' user running hdfs > command in secured mode. I will check using 'hdfs' user is okay for this use > case.
Right now using hdfs user. - Cabir ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27396/#review59451 ----------------------------------------------------------- On Nov. 3, 2014, 8:30 p.m., Cabir Zounaidou wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27396/ > ----------------------------------------------------------- > > (Updated Nov. 3, 2014, 8:30 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, John > Speidel, Mahadev Konar, Nate Cole, and Yusaku Sako. > > > Bugs: AMBARI-1522 > https://issues.apache.org/jira/browse/AMBARI-1522 > > > Repository: ambari > > > Description > ------- > > The idea is to run a scheduler which computes diffs in a component log and > pushes the diff (only if available) to HDFS at fixed interval. > > 1. Each component will have its own monitor > 2. The apscheduler threadpool controls the monitor execution. > 3. If the log directory is not available if the component (or service) is not > running, it will skip the processing. > 4. It saves the last read line index and last modified time for next iteration > 5. It uses HDFS shell utility to push log patch to HDFS. > 6. Right now, the component log directory is configured in a json file. In > the next iteration, it will try to automatically detected from the stack > config. > > > Diffs > ----- > > ambari-agent/src/main/python/ambari_agent/Controller.py dc3a1cf > ambari-agent/src/main/python/ambari_agent/LogSchedulerHandler.py > PRE-CREATION > ambari-agent/src/main/python/ambari_agent/loghandler/__init__.py > PRE-CREATION > ambari-agent/src/main/python/ambari_agent/loghandler/config.py PRE-CREATION > ambari-agent/src/main/python/ambari_agent/loghandler/hdfsapi.py > PRE-CREATION > ambari-agent/src/main/python/ambari_agent/loghandler/monitor.py > PRE-CREATION > ambari-agent/src/main/python/ambari_agent/loghandler/util.py PRE-CREATION > ambari-agent/src/test/python/ambari_agent/TestLogHandler.py PRE-CREATION > > ambari-agent/src/test/python/ambari_agent/dummy_files/log_handler_config.json > PRE-CREATION > > Diff: https://reviews.apache.org/r/27396/diff/ > > > Testing > ------- > > - Ran the ambari-agent with the patch and verified > 1. Started ambari-agent with the patch > 2. The scheduler started successfully > 3. The logs files are getting pushed to HDFS successfully. Verified using > HDFS shell utility. > - Ran the tests successfully > [INFO] > [INFO] > ------------------------------------------------------------------------ > [INFO] Building Ambari Agent 1.3.0-SNAPSHOT > [INFO] > ------------------------------------------------------------------------ > [INFO] > : > ---------------------------------------------------------------------- > Ran 324 tests in 8.282s > > OK > : > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 14.759s > [INFO] Finished at: Thu Oct 30 14:58:21 PDT 2014 > [INFO] Final Memory: 10M/4079M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Cabir Zounaidou > >
