[
https://issues.apache.org/jira/browse/AMBARI-24833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681726#comment-16681726
]
ASF GitHub Bot commented on AMBARI-24833:
-----------------------------------------
oleewere opened a new pull request #19: AMBARI-24833. Re-implement S3/HDFS
outputs as global cloud outputs
URL: https://github.com/apache/ambari-logsearch/pull/19
# What changes were proposed in this pull request?
- ger rid of minio (reason: as it's really abstract solution for s3
backends, the policy handling is done by plain strings, not with objects)
- add hdfs client based and plain s3 outputs (as global configs)
- add hdfs output
- add a lot of new configs for s3 and hdfs
- delete s3 input/output, and hdfs output, delete log spooler framework as
well
Cloud log process implementation:
- create specific loggers for every inputs
- rollover those logs into a different folder (archived)
- new thread which monitors the archive folders, if there is a new file,
upload those to cloud storage
- we will have different configurations, to make sure logs are uploaded
during shutdown (or even startup)
TODOs:
- load hadoop configs from file
- log/archive directories should not be in tmp folder, they should be
configurable
No UTs yet
## How was this patch tested?
Manually with docker env + on local cluster. waiting UTs to pass
Please review @g-boros
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Log Feeder: send logs to cloud storage (gcs/s3 etc.)
> ----------------------------------------------------
>
> Key: AMBARI-24833
> URL: https://issues.apache.org/jira/browse/AMBARI-24833
> Project: Ambari
> Issue Type: Bug
> Components: ambari-logsearch
> Affects Versions: 2.7.0
> Reporter: Olivér Szabó
> Assignee: Olivér Szabó
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.8.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
