Sid,
Validated following on Atlas 2.2.0-rc0 - looks good overall! Found couple of
issues given below; please review.
Thanks,
Madhan
1. successfully built Atlas using docker, for profile external-hbase-solr
(instructions in dev-support/atlas-docker/README.txt)
2. successfully brought up Atlas in docker containers
3. successfully ran quick_start
4. search: performed basic search, advanced search, quick search
5. classification: performed add, associate, propagate, update,
disassociate, delete
6. business-metadata: performed import, add/update/remove on entities, search
7. glossary: performed import, add/remove on entities, searched
8. verified basic operations on HBase hook (create table, drop table)
9. verified basic operations on Hive hook (create database/table/view,
lineage, alter table, drop table/database)
10. verified entity audit entries for above operations
Issues found:
1. dev-support/atlas-docker/.env has BRANCH=branch-2.0; this should be
updated to release-2.2.0-rc0, so that docker build would use Atlas sources from
this tag
2. Atlas startup logs NoSuchFileException for
${atlas.log.dir}/atlas-metrics.out, which looks like incorrect metrics
configuration in hadoop-metrics2.properties.
2021-08-05 07:54:07,353 WARN - [main:] ~ Error creating sink
'atlas-debug-metrics-context' (MetricsSystemImpl:508)
org.apache.hadoop.metrics2.impl.MetricsConfigException: Error creating plugin:
org.apache.hadoop.metrics2.sink.FileSink
at
org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:210)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:531)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSinks(MetricsSystemImpl.java:503)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:479)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:188)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:163)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:62)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:58)
at
org.apache.atlas.web.service.DebugMetricsWrapper.init(DebugMetricsWrapper.java:43)
...
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at
org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:110)
at org.apache.atlas.Atlas.main(Atlas.java:133)
Caused by: org.apache.hadoop.metrics2.MetricsException: Error creating
${atlas.log.dir}/atlas-metrics.out
at org.apache.hadoop.metrics2.sink.FileSink.init(FileSink.java:53)
at
org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:207)
... 63 more
Caused by: java.nio.file.NoSuchFileException: ${atlas.log.dir}/atlas-metrics.out
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at
sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at
java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
at java.nio.file.Files.newOutputStream(Files.java:216)
at org.apache.hadoop.metrics2.sink.FileSink.init(FileSink.java:50)
... 64 more
On 8/4/21, 1:34 PM, "Sidharth Mishra" <[email protected]> wrote:
Atlas team,
Apache Atlas 2.2.0 release candidate #0 is now available for vote
within the dev community. Links to the release artifacts are given
below. Please review and vote.
The vote will be open for at least 72 hours or until necessary votes
are reached.
[ ] +1 Approve
[ ] +0 No opinion
[ ] -1 Disapprove (and reason why)
Thanks,
Sid
List of all issues addressed in this release:
https://issues.apache.org/jira/issues/?jql=project%3DATLAS%20AND%20status%3DResolved%20AND%20fixVersion%3D2.2.0%20ORDER%20BY%20key%20DESC
Git tag for the release:
https://github.com/apache/atlas/tree/release-2.2.0-rc0
Sources for the release:
https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc0/apache-atlas-2.2.0-sources.tar.gz
Source release verification:
PGP Signature:
https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc0/apache-atlas-2.2.0-sources.tar.gz.asc
SHA512 Hash:
https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc0/apache-atlas-2.2.0-sources.tar.gz.sha512
MD5 Hash:
https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc0/apache-atlas-2.2.0-sources.tar.gz.md5
Keys to verify the signature of the release artifacts are available
at: https://dist.apache.org/repos/dist/release/atlas/KEYS
New features/enhancements:
- Introduced Deferred Action - Tag propagation will be handled as
a background process (AtlasTask)
- Added Re-indexing as JAVA_PATCH
- Created JavaPatch to add new super types to existing entities
- Supported for Business Metadata in Atlas Export API
- Supported HA for admin/task API
- Enhanced export service to export Terms
- Added new models and addition attributes to Atlas for Apache
Flink, GCP and ADLS Gen 2
- Added support to Configure HiveServer2 Hook to send Lineage-only
Messages
- Improved and enhanced Bulk Glossary Import supporting import
with relations
- Dependency between Atlas and Kafka Zookeeper removed
- Improved performance of GraphHelper's guid and status getter methods
- Added new privilege - "admin-audits" for Atlas Admin Audits
authorization
- Basic search Improvements - search text with special characters,
optimize pagination
- Updated free-text search processor to support Elasticsearch
- Refactored and enhanced Atlas Python Client supporting Python 2.7
- DSL Search enhancements - Support glossary terms and
relationship, can handle null attribute values, Java Tinkerpop
Traversals instead of GremlinScriptEngine, support search by
classification and its attribute, Caching mechanism for translated dsl
queries
- Enhanced Export/Import Service to conditionally Support
Simultaneous Operations
- Enhanced NotificationHookConsumer for Large Message Processing
- Enhanced Atlas authorization for Add/Update/Remove
classification on entities
- Provided option to add mandatory attribute to existing entity
definition
- Fixed Atlas UI to load faster in case of large number of
classifications & entities
- Updated versions of dependent libraries/components: JanusGraph,
elasticsearch, JQuery, Http core, Http Client, slf4j, log4j, ant,
gremlin, Solr, groovy, netty, Kafka
