[
https://issues.apache.org/jira/browse/ATLAS-3610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17046685#comment-17046685
]
Mariusz Górski commented on ATLAS-3610:
---------------------------------------
Thanks for the information [~kevalbhatt]
We have successfully run Atlas with this patch in both embedded and not
embedded mode in our env.
I noticed, however, how tests should be adjusted so that they do not fail -
you'll find it in ATLAS-3610-3.patch
Re-run tests on localhost and seems the issue is gone:
/usr/lib/jvm/java-8-openjdk-amd64/bin/java
-Dmaven.multiModuleProjectDirectory=/home/mariusz/IdeaProjects/atlas/distro
-Dmaven.home=/snap/intellij-idea-community/208/plugins/maven/lib/maven3
-Dclassworlds.conf=/snap/intellij-idea-community/208/plugins/maven/lib/maven3/bin/m2.conf
-Dmaven.ext.class.path=/snap/intellij-idea-community/208/plugins/maven/lib/maven-event-listener.jar
-javaagent:/snap/intellij-idea-community/208/lib/idea_rt.jar=33041:/snap/intellij-idea-community/208/bin
-Dfile.encoding=UTF-8 -classpath
/snap/intellij-idea-community/208/plugins/maven/lib/maven3/boot/plexus-classworlds-2.6.0.jar
org.codehaus.classworlds.Launcher -Didea.version2019.3.3 test
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------< org.apache.atlas:atlas-distro >--------------------
[INFO] Building Apache Atlas Distribution 3.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @
atlas-distro ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @ atlas-distro ---
[INFO] Executing: /bin/sh -c cd '/home/mariusz/IdeaProjects/atlas/distro' &&
'git' 'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: /home/mariusz/IdeaProjects/atlas/distro
[INFO] Storing buildNumber: 4229cb3c7a9e7109c883431bc5fb502339d077ec at
timestamp: 1582813388815
[INFO] Storing buildScmBranch: master
[INFO]
[INFO] --- apache-rat-plugin:0.13:check (rat-check) @ atlas-distro ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 62 implicit excludes (use -debug for more details).
[INFO] 55 explicit excludes (use -debug for more details).
[INFO] 41 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated:
0, approved: 36 licenses.
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ atlas-distro
---
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @
atlas-distro ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 11 resources to bin
[INFO] Copying 16 resources to conf
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ atlas-distro
---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @
atlas-distro ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/mariusz/IdeaProjects/atlas/distro/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @
atlas-distro ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-distro ---
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ atlas-distro ---
test_jar_java_lookups_fail (TestMetadata.TestMetadata) ... ok
test_jar_java_lookups_succeed_from_path (TestMetadata.TestMetadata) ... ok
test_main_default (TestMetadata.TestMetadata) ... Logging to console is
disabled.
get_config_mock_side_effect (atlas.graph.storage.backend)
Apache Atlas Server started!!!
ok
test_main_embedded (TestMetadata.TestMetadata) ... Logging to console is
disabled.
get_config_mock_side_effect (atlas.graph.storage.backend)
configured for local hbase.
hbase started.
configured for local solr.
get_config_mock_side_effect (atlas.graph.index.search.solr.zookeeper-url)
solr started.
setting up solr collections...
Apache Atlas Server started!!!
ok
----------------------------------------------------------------------
Ran 4 tests in 0.005s
OK
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 5.710 s
[INFO] Finished at: 2020-02-27T15:23:12+01:00
[INFO] ------------------------------------------------------------------------
Thanks for the support and patience with this !
cc [~mehul] [~vishal.suvagia] [~Rahul_FI]
> Enable logging to multiple targets (logdir, stdout)
> ---------------------------------------------------
>
> Key: ATLAS-3610
> URL: https://issues.apache.org/jira/browse/ATLAS-3610
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 3.0.0
> Reporter: Mariusz Górski
> Priority: Minor
> Fix For: 2.1.0, 3.0.0
>
> Attachments: ATLAS-3610-1.patch, ATLAS-3610-2.patch, ATLAS-3610.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> *Problem description*
> When starting Atlas, Python scripts are invoked to spawn all necessary
> processes. This is achieved with the use of _subprocess.Popen()_ in
> _runProcess()_ function of _atlas_config.py_ script. While doing that, stdout
> is being redirected to log files, which (for java processes) can override
> log4j configuration (if configured with stdout handler). This creates
> situation when log4j configuration cannot be relied upon and prevents logging
> to stdout.
> *Proposed solution*
> * Add possibility to log to stdout by using multiple stdout/stderr handlers
> * Adjust _runProcess()_ function from _atlas_config.py_ script (and all
> functions relying on it) to provide _logconsole_ kwarg which will add
> additional handler to Popen stdout/stderr redirection
> * Add _ENABLE_LOGGING_TO_CONSOLE_ env variable (defaulting to _False_) for
> backwards compatibility.
> * Adjust _atlas_start.py_ script to make use of _ENABLE_LOGGING_TO_CONSOLE_
> and provide it in functions invoking atlas/solr/elasticsearch/zookeeper
> *Predicted Benefits*
> This improvement would bring the benefit of enabling logging to stdout (which
> is supressed for now even if configured in spawned components), which could
> be very useful when running Atlas in docker container or in Kubernetes/OC,
> where some logging scraping is present.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
