Re: Review Request 56724: ATLAS-1390: Add IBM Graph Support
the fork source. graphdb/ibm-graph/readme.txt Lines 72 (patched) <https://reviews.apache.org/r/56724/#comment239447> Remove "This is happening because the" i.e. just start the sentence with "EntityLineageService is hard-coded..." graphdb/ibm-graph/readme.txt Lines 73 (patched) <https://reviews.apache.org/r/56724/#comment239452> Do a global replace of "Tinker Pop" with "TinkerPop" graphdb/ibm-graph/readme.txt Lines 79 (patched) <https://reviews.apache.org/r/56724/#comment239449> Remove "The" graphdb/ibm-graph/readme.txt Lines 94 (patched) <https://reviews.apache.org/r/56724/#comment239485> "This dependency needs to be removed". - David Kantor On Feb. 22, 2017, 11:44 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56724/ > --- > > (Updated Feb. 22, 2017, 11:44 p.m.) > > > Review request for atlas, David Kantor and Neeru Gupta. > > > Bugs: ATLAS-1390 > https://issues.apache.org/jira/browse/ATLAS-1390 > > > Repository: atlas > > > Description > --- > > This work item adds IBM Graph as a graph database that can be used by Atlas. > There are some limitations of this implementation. These are documented in > the readme.txt under graphdb\ibm-graph. The code here is the result of many > months of development effort. It was contributed to by myself, Dave Kantor, > Neeru Gupta, and Marcin Lewandowski. It was developed on our private fork of > Atlas and is now being added to the public Atlas so that others can benefit > from it. > > IBM Graph is a cloud based graph based on Titan 1. It is a little different > from Titan though in that all of the interactions with it requires calling a > REST API. This implementation is optimized to make as few REST API calls as > possible by combining the update operations into a single Gremlin script that > gets sent when the transaction is committed. It also has full multi-tenancy > support, although in this JIRA we have not put in the plumbing that allows > Atlas to make use of it. For the mult-user support, each thread has its own > transaction associated with it. We followed the Titan semantics -- vertex > queries executed in the context of a transaction take into account > uncommitted changes. Pure gremlin queries do not, and will only reflect > committed changes. > > IBM Graph also has some restrictions that a normal Titan instance would not. > Because it is a shared environment, there is a restriction that only queries > with an initial indexed predicate will run. Any query that requires a full > graph scan is rejected. Many of the tests in Atlas attempt to do a graph > scan. These tests have been changed so that they are skipped when ibm-graph > is being used as the graph backend. > > > Note that this implementation depends on Tinkerpop 3 classes, which require > Java 8. To manage the java version dependency, we're following a similar > route to what we did for Titan 1 support. If Java 7 is being used, the > ibm-graph code is not compiled, but we still create a jar file which is > basically empty. This is needed so that there is a jar in place to satisfy > the maven dependency on the ibm-graph implementation in the graphdb-impls > project. If Java 8 is being used, the ibm-graph code is compiled. The tests > in the ibm-graph project are only run if the ibm-graph profile is enabled > though. > > By default, Titan 0.5.4 is still used by Atlas. To use the ibm-graph > implemention, you need to run with java 8, enable the ibm-graph profile, and > add properties to your settings.xml to configure the IBM Graph service. This > is described in the readme. > > > Diffs > - > > common/src/main/java/org/apache/atlas/GraphInitializationException.java > PRE-CREATION > common/src/main/java/org/apache/atlas/groovy/LiteralExpression.java > 14074994f7fe06fc6a076cedb833a7de2f85e096 > > common/src/main/java/org/apache/atlas/groovy/VariableAssignmentExpression.java > 1aa74435d59dfb4d8d3077febe794c5b1780a824 > distro/pom.xml c0090ae2a86faa0b697bbdd1156323ac9f944772 > distro/src/conf/atlas-application.properties > b2b8e74590c572a0a63f145e1ac9c6a3cc38f677 > distro/src/conf/credentials.json PRE-CREATION > distro/src/main/assemblies/standalone-package.xml > 215cb2368cf15f5fc0bd3ece19f2db330cc56b6b > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java > f7e5c5cec1bf134e58ff5de8986a38580fe082e3 > >
[jira] [Resolved] (ATLAS-499) Automatically update inverse references to prevent repository corruption
[ https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor resolved ATLAS-499. Resolution: Fixed Fix Version/s: 0.8-incubating > Automatically update inverse references to prevent repository corruption > > > Key: ATLAS-499 > URL: https://issues.apache.org/jira/browse/ATLAS-499 > Project: Atlas > Issue Type: Improvement > Reporter: David Kantor > Assignee: David Kantor > Fix For: 0.8-incubating > > > For references that are defined in the type system as bi-directional, Atlas > does not enforce that clients update both sides of the reference, and relies > on well behaved client applications for repository consistency. This allows > the repository to be easily corrupted with unbalanced references. Atlas > should automatically update the inverse reference to avoid this corruption. > It also is a performance optimization, in that clients can update a > bi-directional non-composite reference without retrieving both entities, the > only need the guid for the target entity. > Note that changes are being tracked/applied separately for the V1/legacy and > V2 code paths under the sub-tasks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1552: Attachment: rb56860.patch Rebased with master. > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > Fix For: 0.8-incubating > > Attachments: rb56860.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1552: Attachment: (was: rb56860.patch) > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/ --- (Updated March 1, 2017, 2:18 p.m.) Review request for atlas. Changes --- Rebase with master Bugs: ATLAS-1552 https://issues.apache.org/jira/browse/ATLAS-1552 Repository: atlas Description --- Automatic update of inverse references in V2 (V1?) code path. Diffs (updated) - intg/src/main/java/org/apache/atlas/type/AtlasStructType.java bb7eef88a44c6292a9b51c72dd530dba5ffa274e intg/src/test/java/org/apache/atlas/TestUtilsV2.java 0b28bcf1ebba5fa6f936964934bf60caee91fa3b intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java e2b82ccd89eb2053b633f7f070b1744dc5755299 repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java aab0d387e97f36541bf9f935e8410ff87aed repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION Diff: https://reviews.apache.org/r/56860/diff/4/ Changes: https://reviews.apache.org/r/56860/diff/3-4/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
> On Feb. 27, 2017, 5:27 p.m., David Radley wrote:
> > I have done a little testing at V2 to model a parent children hierarchy
> > that we would need to define a Glossary (see Jira 1410)
> >
> > I define types EntityA with a children 1 to many optional attribute to
> > EntityB. EntityB has a mantidory pointer to an EntityA parent.
> >
> > EntityA type has
> > "name": "children",
> > "typeName": "array",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "ownedRef"
> > }
> > ],
> > ],
> > "isIndexable": false,
> > "isOptional": true,
> > "isUnique": false
> > },
> > EntityB type has attributes like this:
> > "name": "parent",
> > "typeName": "EntityA",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "inverseRef",
> > "params": {
> > "attribute": "children"
> > }
> > }
> > ],
> > "isIndexable": false,
> > "isOptional": false,
> > "isUnique": false
> >
> > I then define
> > -an instance of EntityA
> > -an instance of EntityB with parent pointing to EntityA (I have to supply
> > the parent attribute as it is isOptional false).
> >
> > If I then get (using the API) the EntityA instance it does not have any
> > content in its children. Is this expected? It seems that if I am allowed to
> > do this then the children and parent pointers are out of sync.
> >
> > Am I missing something?
I added a test case that mimics your use case and could not reproduce the issue
- when I get the EntityA instance its children attribute contains the EntityB
instance, indicating the inverse reference was updated correctly. Perhaps you
were running without my changes? I will follow up with you.
- David
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166897
---
On Feb. 24, 2017, 3 p.m., David Kantor wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> ---
>
> (Updated Feb. 24, 2017, 3 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java
> 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java
> 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java
> 2d71d4135025339cd072982efa1231575bf21035
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
> 3f87e695b01a21f0cbd453f6de68269c5fe025aa
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> f1efe46404d71187c2691f9b795916ed6c26ef3e
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
> 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java
> 555f0acc22c4bec67f1a2a4064acad66e939747a
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java
> PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> ---
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
[jira] [Comment Edited] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883414#comment-15883414 ] David Kantor edited comment on ATLAS-1552 at 2/24/17 8:01 PM: -- All review comments have been addressed, attaching updated patch. This patch will be committed EOD Monday Feb 27 unless there are further comments on the review board request. was (Author: dkantor): All review comments have been addressed, attaching updated patch. This patch will be committed EOD Monday Feb 24 unless there are further comments on the review board request. > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > Attachments: rb56860.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1552: Attachment: rb56860.patch All review comments have been addressed, attaching updated patch. This patch will be committed EOD Monday Feb 24 unless there are further comments on the review board request. > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > Attachments: rb56860.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1552: Attachment: (was: rb56860.patch) > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/ --- (Updated Feb. 24, 2017, 3 p.m.) Review request for atlas. Changes --- Fix log messages Bugs: ATLAS-1552 https://issues.apache.org/jira/browse/ATLAS-1552 Repository: atlas Description --- Automatic update of inverse references in V2 (V1?) code path. Diffs (updated) - intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684 intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION Diff: https://reviews.apache.org/r/56860/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
> On Feb. 23, 2017, 7:14 a.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java, > > line 333 > > <https://reviews.apache.org/r/56860/diff/1/?file=1644998#file1644998line333> > > > > It might be useful to add a method in AtlasAttribute, to return reverse > > attribute's AtlasAttribute. This can be initialized in > > AtlasEntityType.resolveReferencesPhase2(). > > > > This will help avoid lookups during entity attribute updates. Good idea, I have applied the suggested changes. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/#review166483 ------- On Feb. 24, 2017, 2:25 p.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56860/ > --- > > (Updated Feb. 24, 2017, 2:25 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1552 > https://issues.apache.org/jira/browse/ATLAS-1552 > > > Repository: atlas > > > Description > --- > > Automatic update of inverse references in V2 (V1?) code path. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/type/AtlasStructType.java > 57ad1063eb0d225102a2b8b72232054f57b9c36b > intg/src/test/java/org/apache/atlas/TestUtilsV2.java > 84e17cb364fbf6d07b2211d85e336d8146fdb684 > intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java > 2d71d4135025339cd072982efa1231575bf21035 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java > 3f87e695b01a21f0cbd453f6de68269c5fe025aa > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java > f1efe46404d71187c2691f9b795916ed6c26ef3e > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java > 09f69dbe6f519d84ae8e223532fc5adaa411ed9b > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java > 555f0acc22c4bec67f1a2a4064acad66e939747a > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/56860/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/ --- (Updated Feb. 24, 2017, 2:25 p.m.) Review request for atlas. Changes --- Address review comments Bugs: ATLAS-1552 https://issues.apache.org/jira/browse/ATLAS-1552 Repository: atlas Description --- Automatic update of inverse references in V2 (V1?) code path. Diffs (updated) - intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684 intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION Diff: https://reviews.apache.org/r/56860/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
> On Feb. 23, 2017, 7:23 p.m., Suma Shivaprasad wrote: > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java, > > line 330 > > <https://reviews.apache.org/r/56860/diff/1/?file=1644999#file1644999line330> > > > > handle map case as well? Maps are specifically not supported for inverse reference updating, in both the legacy code I delivered recently for ATLAS-1551 and these changes - if the inverse reference is a map, what would be used for the key when adding the map entry? The map attribute itself can have a inverse reference, and if that inverse reference is not a map, it will be automatically updated. I will add a test to verify that use case. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/#review166552 --- On Feb. 22, 2017, 11:48 p.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56860/ > --- > > (Updated Feb. 22, 2017, 11:48 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1552 > https://issues.apache.org/jira/browse/ATLAS-1552 > > > Repository: atlas > > > Description > --- > > Automatic update of inverse references in V2 (V1?) code path. > > > Diffs > - > > intg/src/test/java/org/apache/atlas/TestUtilsV2.java > 84e17cb364fbf6d07b2211d85e336d8146fdb684 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java > 49d5a080158e3e54fd917a13f743f1cf495e66b0 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java > 09f69dbe6f519d84ae8e223532fc5adaa411ed9b > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java > 555f0acc22c4bec67f1a2a4064acad66e939747a > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/56860/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
Re: Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
> On Feb. 23, 2017, 7:14 a.m., Madhan Neethiraj wrote:
> > Consider the following scenario:
> > - create hive_table 'employee'
> > - create hive_column 'name'; set attribute 'table' to 'employee' table
> > created above
> > - now 'employee.columns' attribute lists 'name' - good!
> > - create hive_table 'person'
> > - edit the hive_column created above ('name'), to set attribute 'table' to
> > 'person' table
> > - now 'person.columns' attribute lists 'name' - good!
> >
> > However, 'name' attribute is still seen in 'employee' table. This should
> > not be the case, right? Can you please review?
I think what you are seeing in this use case is the result of having soft
deletes enabled, and based on my understanding of how soft deletion is
implemented, it is the expected behavior. The graph edge which represents the
reference from hive_table(employee).columns to hive_column(name) is in fact
deleted. With soft deletes enabled, deleting an edge or vertex amounts to
setting its __state property to DELETED. However, the edge ID for the deleted
edge is *not* deleted from the hive_table(employee).columns property value -
see the comments in DeleteHandlerV1.deleteEdgeBetweenVertices() lines 465-467
(which were copied from DeleteHandler.deleteEdgeBetweenVertices() lines
349-351). When soft deletes are enabled, shouldUpdateReverseAttribute is
false, so the logic to remove the edge ID for the deleted edge from the
reference property value is not executed, and the edge ID for the deleted edge
remains in the property. When the entity is retrieved, there is no logic that
I could find
which filters out soft deleted edges for a multiplicity-many reference (i.e.
an attribute that is an array of CLASS (legacy) or OBJECT_ID_TYPE (new)
elements) in either the legacy code path (GraphToTypedInstanceMapper) or in the
new code path (EntityGraphRetriever). The specific logic to not delete the
edge ID from the vertex property (and soft deletes in general) was implemented
by Shwetha Shivalingamurthy. Perhaps she could comment on this.
- David
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166483
-------
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> ---
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java
> 84e17cb364fbf6d07b2211d85e336d8146fdb684
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java
> 49d5a080158e3e54fd917a13f743f1cf495e66b0
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
> 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java
> 555f0acc22c4bec67f1a2a4064acad66e939747a
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java
> PRE-CREATION
>
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java
> PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> ---
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
[jira] [Updated] (ATLAS-1552) Auto reverse reference update in V2 code path
[ https://issues.apache.org/jira/browse/ATLAS-1552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1552: Attachment: rb56860.patch > Auto reverse reference update in V2 code path > - > > Key: ATLAS-1552 > URL: https://issues.apache.org/jira/browse/ATLAS-1552 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.8-incubating > Reporter: David Kantor >Assignee: David Kantor > Attachments: rb56860.patch > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Review Request 56860: ATLAS-1552 auto update of inverse references in V2 code path
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56860/ --- Review request for atlas. Bugs: ATLAS-1552 https://issues.apache.org/jira/browse/ATLAS-1552 Repository: atlas Description --- Automatic update of inverse references in V2 (V1?) code path. Diffs - intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION Diff: https://reviews.apache.org/r/56860/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56856: ATLAS-1535 - Resolve failures in TaxonomyServiceTest
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56856/#review166105 --- Ship it! Ship It! - David Kantor On Feb. 20, 2017, 7:03 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56856/ > --- > > (Updated Feb. 20, 2017, 7:03 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1535 > https://issues.apache.org/jira/browse/ATLAS-1535 > > > Repository: atlas > > > Description > --- > > This resolves the test failures in TaxonomyServiceTest. The service no > longer needs to explicitly manage its transactions. The test has been > updated to reflect this. > > > Diffs > - > > > webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java > 45df9c7c6d55826c551e8b54e52b22be3a63bac0 > > Diff: https://reviews.apache.org/r/56856/diff/ > > > Testing > --- > > TaxonomyServiceTest now passes. > > > Thanks, > > Jeff Hagelberg > >
Re: Review Request 56289: Atlas-695: Add Titan1 suport in Atlas.
> On Feb. 17, 2017, 9:59 p.m., David Kantor wrote: > > graphdb/titan1/readme.txt, line 30 > > <https://reviews.apache.org/r/56289/diff/5/?file=1638465#file1638465line30> > > > > "Configure the Atlas runtime to use Titan 1 by setting the > > atlas.graphdb.backend property, as follows:" "Configure the Atlas runtime to use Titan 1 by setting the atlas.graphdb.backend property in ATLAS_HOME/conf/atlas-application.properties, as follows:" - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56289/#review165979 --- On Feb. 17, 2017, 4 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56289/ > --- > > (Updated Feb. 17, 2017, 4 p.m.) > > > Review request for atlas, David Kantor and Jeff Hagelberg. > > > Bugs: ATLAS-695 > https://issues.apache.org/jira/browse/ATLAS-695 > > > Repository: atlas > > > Description > --- > > ATLAS-695 Add titan1 project in atlas > > > Diffs > - > > catalog/pom.xml 00e56dc6d01959f81a2d0884396837e261cb3ff0 > catalog/src/main/java/org/apache/atlas/catalog/query/BaseQuery.java > 6a2ba53ddfc740bf77baf319b458853bd33864ec > > catalog/src/test/java/org/apache/atlas/catalog/query/AtlasEntityQueryTest.java > 901a549a8945d7314373feac8b9a5fea1340a85d > graphdb/graphdb-impls/pom.xml 57b5137178dd179f86edb9a7af4f260b4b7d0ece > graphdb/pom.xml 2d303e4d5c29943ceef7883f3ad10a366d7f26aa > graphdb/titan0/pom.xml 9d88a72f60a3fa6969a94bf62f40d0192781c199 > > graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java > 75ea54511535ab72b5d00e7c5d8a1eccaffc3ef2 > > graphdb/titan0/src/main/java/org/apache/atlas/utils/IteratorToIterableAdapter.java > ed7c2a71ed1f12470a0a19e475adb145282c1681 > graphdb/titan1/pom.xml PRE-CREATION > graphdb/titan1/readme.txt PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/GraphDbObjectFactory.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Edge.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Element.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Graph.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphDatabase.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphIndex.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphManagement.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1IndexQuery.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1PropertyKey.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Vertex.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1VertexQuery.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/TitanObjectFactory.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/graphson/AtlasElementPropertyConfig.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/graphson/AtlasGraphSONMode.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/graphson/AtlasGraphSONTokens.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/graphson/AtlasGraphSONUtility.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/NativeTitan1GraphQuery.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/Titan1GraphQuery.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/serializer/BigDecimalSerializer.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/serializer/BigIntegerSerializer.java > PRE-CREATION > > grap
Re: Review Request 56289: Atlas-695: Add Titan1 suport in Atlas.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56289/#review165979 --- graphdb/titan1/readme.txt (line 11) <https://reviews.apache.org/r/56289/#comment237850> * "The configured graph backend is determined by maven profile. The default profile is titan0. To build Atlas configured to run against Titan 1, the titan1 profile must be enabled." * This reads more like an issues list, not architecture notes. I expected mention of how there is a TP3 gremlin translator used when Titan 1 is configured, and any other such architectural/design info. Move the issues to the KNOWN ISSUES section. graphdb/titan1/readme.txt (line 22) <https://reviews.apache.org/r/56289/#comment237844> Remove "First, you need to". Suggested alternative wording: "Build Atlas with the titan1 maven profile enabled". graphdb/titan1/readme.txt (line 24) <https://reviews.apache.org/r/56289/#comment237843> Are there any known test failures with titan 1? If so, wouldn't -Dmaven.test.failure.ignore be needed for the build to complete? graphdb/titan1/readme.txt (line 30) <https://reviews.apache.org/r/56289/#comment237845> "Configure the Atlas runtime to use Titan 1 by setting the atlas.graphdb.backend property, as follows:" graphdb/titan1/readme.txt (line 34) <https://reviews.apache.org/r/56289/#comment237846> No emoticons in readmes. And saying "hope that the webapp issues have been fixed" is not useful. Alternative: "Attempt to start the Atlas server. NOTE: As of this writing, Atlas fails to start (see issue 2 below)." graphdb/titan1/readme.txt (line 40) <https://reviews.apache.org/r/56289/#comment237848> Is there a JIRA for the specific changes described here? If so, mentioned the JIRA. If not, create one and mention it here. graphdb/titan1/readme.txt (line 42) <https://reviews.apache.org/r/56289/#comment237847> The catalog project... Is there a JIRA for the specific changes described here? If so, mentioned the JIRA. If not, create one and mention it here. graphdb/titan1/readme.txt (line 44) <https://reviews.apache.org/r/56289/#comment237849> Is there a JIRA for the specific changes described here? If so, mentioned the JIRA. If not, create one and mention it here. - David Kantor On Feb. 17, 2017, 4 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56289/ > ------- > > (Updated Feb. 17, 2017, 4 p.m.) > > > Review request for atlas, David Kantor and Jeff Hagelberg. > > > Bugs: ATLAS-695 > https://issues.apache.org/jira/browse/ATLAS-695 > > > Repository: atlas > > > Description > --- > > ATLAS-695 Add titan1 project in atlas > > > Diffs > - > > catalog/pom.xml 00e56dc6d01959f81a2d0884396837e261cb3ff0 > catalog/src/main/java/org/apache/atlas/catalog/query/BaseQuery.java > 6a2ba53ddfc740bf77baf319b458853bd33864ec > > catalog/src/test/java/org/apache/atlas/catalog/query/AtlasEntityQueryTest.java > 901a549a8945d7314373feac8b9a5fea1340a85d > graphdb/graphdb-impls/pom.xml 57b5137178dd179f86edb9a7af4f260b4b7d0ece > graphdb/pom.xml 2d303e4d5c29943ceef7883f3ad10a366d7f26aa > graphdb/titan0/pom.xml 9d88a72f60a3fa6969a94bf62f40d0192781c199 > > graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java > 75ea54511535ab72b5d00e7c5d8a1eccaffc3ef2 > > graphdb/titan0/src/main/java/org/apache/atlas/utils/IteratorToIterableAdapter.java > ed7c2a71ed1f12470a0a19e475adb145282c1681 > graphdb/titan1/pom.xml PRE-CREATION > graphdb/titan1/readme.txt PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/GraphDbObjectFactory.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Edge.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Element.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Graph.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphDatabase.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphIndex.java > PRE-CREATION > > graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1GraphManage
Re: Review Request 56535: Fix DeleteHandlerV1 for new model changes and add tests
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56535/#review165870 --- Ship it! Ship It! - David Kantor On Feb. 16, 2017, 6:55 a.m., Suma Shivaprasad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56535/ > --- > > (Updated Feb. 16, 2017, 6:55 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1547 > https://issues.apache.org/jira/browse/ATLAS-1547 > > > Repository: atlas > > > Description > --- > > Added UTs for delete operations > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java > 34c10f4 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java > 4d11b20 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java > b35d288 > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1Test.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1Test.java > 987951e > server-api/src/main/java/org/apache/atlas/RequestContextV1.java bf73174 > > Diff: https://reviews.apache.org/r/56535/diff/ > > > Testing > --- > > Added tests for hard delete and soft delete > > > Thanks, > > Suma Shivaprasad > >
[jira] [Updated] (ATLAS-1539) webapp tests fail when run from the webapp project directory
[
https://issues.apache.org/jira/browse/ATLAS-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1539:
Attachment: rb56463.patch
Rebased, addressed review comments, attached updated patch.
> webapp tests fail when run from the webapp project directory
>
>
> Key: ATLAS-1539
> URL: https://issues.apache.org/jira/browse/ATLAS-1539
> Project: Atlas
> Issue Type: Bug
> Reporter: David Kantor
> Assignee: David Kantor
> Attachments: rb56463.patch
>
>
> Integration tests in webapp mostly fail when the tests are invoked from the
> webapp project directory. The jetty container used by the tests is
> compromised due to the error shown below, because the
> atlas-application.properties file in the typesystem test jar has properties
> which assume the tests will always be run from the top level e.g.
> {noformat}
> atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
> atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
> {noformat}
> Here is the error which occurs within jetty:
> {noformat}
> 2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
> file,
> filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
> (UserDao:74)
> java.io.FileNotFoundException:
> C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The
> system cannot find the path specified)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
> at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (ATLAS-1539) webapp tests fail when run from the webapp project directory
[
https://issues.apache.org/jira/browse/ATLAS-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1539:
Attachment: (was: rb56463.patch)
> webapp tests fail when run from the webapp project directory
>
>
> Key: ATLAS-1539
> URL: https://issues.apache.org/jira/browse/ATLAS-1539
> Project: Atlas
> Issue Type: Bug
> Reporter: David Kantor
> Assignee: David Kantor
>
> Integration tests in webapp mostly fail when the tests are invoked from the
> webapp project directory. The jetty container used by the tests is
> compromised due to the error shown below, because the
> atlas-application.properties file in the typesystem test jar has properties
> which assume the tests will always be run from the top level e.g.
> {noformat}
> atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
> atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
> {noformat}
> Here is the error which occurs within jetty:
> {noformat}
> 2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
> file,
> filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
> (UserDao:74)
> java.io.FileNotFoundException:
> C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The
> system cannot find the path specified)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
> at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 12, 2017, 7:48 p.m.) Review request for atlas. Changes --- Addressed Madhan's review comments. Added ApplicationPropertiesTest. Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files in the typesystem src/test/resources rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 common/src/main/java/org/apache/atlas/ApplicationProperties.java 9b1e9cd48a52da1cc388a808a67817e41741209d common/src/test/java/org/apache/atlas/ApplicationPropertiesTest.java PRE-CREATION common/src/test/resources/test.properties PRE-CREATION typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 12, 2017, 5:47 p.m.) Review request for atlas. Changes --- Remove whitespace Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files in the typesystem src/test/resources rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 common/src/main/java/org/apache/atlas/ApplicationProperties.java 9b1e9cd48a52da1cc388a808a67817e41741209d typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
> On Feb. 10, 2017, 8:46 p.m., Jeff Hagelberg wrote: > > authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java, > > line 125 > > <https://reviews.apache.org/r/56463/diff/4/?file=1628058#file1628058line125> > > > > Is it worth documenting this new behavior anywhere? Documented in javadoc of ApplicationProperties.getFileAsInputStream. > On Feb. 10, 2017, 8:46 p.m., Jeff Hagelberg wrote: > > authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java, > > line 160 > > <https://reviews.apache.org/r/56463/diff/4/?file=1628058#file1628058line160> > > > > Should we use the Thread's context classloader here? Fixed > On Feb. 10, 2017, 8:46 p.m., Jeff Hagelberg wrote: > > webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java, line 66 > > <https://reviews.apache.org/r/56463/diff/4/?file=1628063#file1628063line66> > > > > This seems to largely duplicate the logic above, and my comment about > > the classloader applies here as well. Can the logic be moved to some > > utility method? It seems like the only thing different is the names of the > > properties being used and the default value. Great idea, refactored logic into new utility method ApplicationProperties.getFileAsInputStream() - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/#review165171 --- On Feb. 12, 2017, 5:41 p.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56463/ > --- > > (Updated Feb. 12, 2017, 5:41 p.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1539 > https://issues.apache.org/jira/browse/ATLAS-1539 > > > Repository: atlas > > > Description > --- > > Integration tests in projects which use the typesystem test jar (e.g. webapp) > were failing when the tests were invoked in the project directory rather than > the top level project, because properties for user-credentials.properties and > policy store files where using a path relative to the working directory, > which amounted to a hard-coded assumption that the tests would only ever be > run from the top level. > Fix: Attempt to load policy store and user credential files as classloader > resource if configured file path does not exist, in the same manner that the > atlas-application.properties is located. Use test copies of these files in > the typesystem src/test/resources rather than a path relative the top level > maven project. > > > Diffs > - > > > authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java > 36db7007a14c2afbda7582409e5dfa474fa9a7b6 > > authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java > 4f9255a3dee5ac59dcef95e76eae423445be6bdd > > authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java > d6e785366a5fd4dc60496d37b2639ebafa06d087 > common/src/main/java/org/apache/atlas/ApplicationProperties.java > 9b1e9cd48a52da1cc388a808a67817e41741209d > typesystem/src/test/resources/atlas-application.properties > 0e6bc4187ad064820d11da137cee3b8b1857560e > typesystem/src/test/resources/policy-store.txt PRE-CREATION > typesystem/src/test/resources/users-credentials.properties PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java > 254d836e5389780b6d10a8d1a664a96b06ad8d1b > > Diff: https://reviews.apache.org/r/56463/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 12, 2017, 5:41 p.m.) Review request for atlas. Changes --- Addressed Jeff's review comments. Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files in the typesystem src/test/resources rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 common/src/main/java/org/apache/atlas/ApplicationProperties.java 9b1e9cd48a52da1cc388a808a67817e41741209d typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
> On Feb. 9, 2017, 7:01 p.m., Apoorv Naik wrote: > > I see that atlas has multiple ways of loading properties/configuration, do > > you think it'd be a better idea to introduce a property/config framework > > (maybe in an enhancement JIRA) that produces apache commons configuration > > object or java properties object for more streamlined use. I've seen 3 > > mixes of config > > > > 1. Load directly into property files > > 2. Use spring bean placeholder > > 3. Read into apache commons configuration I agree that a introducing a property/config framework is a good idea. However, I think that is beyond the scope of this change. For now, I am trying to apply a smaller short term change that addresses the issue described in this Jira and also follow the pattern for other Atlas config files that check the use atlas.conf and the classpath. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/#review164967 ------- On Feb. 9, 2017, 12:47 a.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56463/ > --- > > (Updated Feb. 9, 2017, 12:47 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-1539 > https://issues.apache.org/jira/browse/ATLAS-1539 > > > Repository: atlas > > > Description > --- > > Integration tests in projects which use the typesystem test jar (e.g. webapp) > were failing when the tests were invoked in the project directory rather than > the top level project, because properties for user-credentials.properties and > policy store files where using a path relative to the working directory, > which amounted to a hard-coded assumption that the tests would only ever be > run from the top level. > Fix: Attempt to load policy store and user credential files as classloader > resource if configured file path does not exist, in the same manner that the > atlas-application.properties is located. Use test copies of these files in > the typesystem src/test/resources rather than a path relative the top level > maven project. > > > Diffs > - > > > authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java > 36db7007a14c2afbda7582409e5dfa474fa9a7b6 > > authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java > 4f9255a3dee5ac59dcef95e76eae423445be6bdd > > authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java > d6e785366a5fd4dc60496d37b2639ebafa06d087 > typesystem/src/test/resources/atlas-application.properties > 0e6bc4187ad064820d11da137cee3b8b1857560e > typesystem/src/test/resources/policy-store.txt PRE-CREATION > typesystem/src/test/resources/users-credentials.properties PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java > 254d836e5389780b6d10a8d1a664a96b06ad8d1b > > Diff: https://reviews.apache.org/r/56463/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
Re: Review Request 56287: ATLAS-1551 Auto update of reverse reference in V1 API
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56287/ --- (Updated Feb. 10, 2017, 8:29 p.m.) Review request for atlas. Bugs: ATLAS-1551 https://issues.apache.org/jira/browse/ATLAS-1551 Repository: atlas Description (updated) --- ATLAS-1551 auto update of reverse references in V1 API Diffs - repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java e2bc02871714907af35fd3acbf73b39094b65d72 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java f18c308346034a475c65b7103988d542dd8b1796 repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java PRE-CREATION typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java 5c977c147a63106b810f8855f3e7905b4075e165 Diff: https://reviews.apache.org/r/56287/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56287: ATLAS-1551 Auto update of reverse reference in V1 API
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56287/ --- (Updated Feb. 10, 2017, 8:28 p.m.) Review request for atlas. Summary (updated) - ATLAS-1551 Auto update of reverse reference in V1 API Bugs: ATLAS-1551 https://issues.apache.org/jira/browse/ATLAS-1551 Repository: atlas Description --- ATLAS-499 auto update of reverse references in V1 API Diffs - repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java e2bc02871714907af35fd3acbf73b39094b65d72 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java f18c308346034a475c65b7103988d542dd8b1796 repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java PRE-CREATION typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java 5c977c147a63106b810f8855f3e7905b4075e165 Diff: https://reviews.apache.org/r/56287/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56287: ATLAS-499 Auto update of reverse reference
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56287/ --- (Updated Feb. 10, 2017, 8:28 p.m.) Review request for atlas. Changes --- Changed bug to ATLAS-1551, which is the sub-task of ATLAS-499 that is specifically tracking the V1 changes. Bugs: ATLAS-1551 https://issues.apache.org/jira/browse/ATLAS-1551 Repository: atlas Description --- ATLAS-499 auto update of reverse references in V1 API Diffs - repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java e2bc02871714907af35fd3acbf73b39094b65d72 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java f18c308346034a475c65b7103988d542dd8b1796 repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java PRE-CREATION typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java 5c977c147a63106b810f8855f3e7905b4075e165 Diff: https://reviews.apache.org/r/56287/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
[jira] [Updated] (ATLAS-499) Automatically update inverse references to prevent repository corruption
[ https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-499: --- Description: For references that are defined in the type system as bi-directional, Atlas does not enforce that clients update both sides of the reference, and relies on well behaved client applications for repository consistency. This allows the repository to be easily corrupted with unbalanced references. Atlas should automatically update the inverse reference to avoid this corruption. It also is a performance optimization, in that clients can update a bi-directional non-composite reference without retrieving both entities, the only need the guid for the target entity. Note that changes are being tracked/applied separately for the V1/legacy and V2 code paths under the sub-tasks. was:For references that are defined in the type system as bi-directional, Atlas does not enforce that clients update both sides of the reference, and relies on well behaved client applications for repository consistency. This allows the repository to be easily corrupted with unbalanced references. Atlas should automatically update the inverse reference to avoid this corruption. > Automatically update inverse references to prevent repository corruption > > > Key: ATLAS-499 > URL: https://issues.apache.org/jira/browse/ATLAS-499 > Project: Atlas > Issue Type: Improvement > Reporter: David Kantor > Assignee: David Kantor > > For references that are defined in the type system as bi-directional, Atlas > does not enforce that clients update both sides of the reference, and relies > on well behaved client applications for repository consistency. This allows > the repository to be easily corrupted with unbalanced references. Atlas > should automatically update the inverse reference to avoid this corruption. > It also is a performance optimization, in that clients can update a > bi-directional non-composite reference without retrieving both entities, the > only need the guid for the target entity. > Note that changes are being tracked/applied separately for the V1/legacy and > V2 code paths under the sub-tasks. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 47658: ATLAS-499: Update reverse references
> On Feb. 10, 2017, 7:50 p.m., Madhan Neethiraj wrote: > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java, > > line 749 > > <https://reviews.apache.org/r/47658/diff/8/?file=1421934#file1421934line749> > > > > Isn't there a need for "removeReverseReference()" - to update the > > reverseAttribute when a referred entity is deleted? For example: when a > > hive_column entity is deleted, hive_table.columns needs to be updated. > > > > Perhaps this is already handled in the current implementation. If so, > > please ignore this comment. This is already handled in DeleteHandler.deleteEdge(). Also this review request had been discarded a few days ago, so that I could separate the V1 and V2 changes. If you have any further review comments on the V1 changes, please post them to https://reviews.apache.org/r/56287/ - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47658/#review165156 ------- On June 17, 2016, 2:44 a.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47658/ > --- > > (Updated June 17, 2016, 2:44 a.m.) > > > Review request for atlas and Shwetha GS. > > > Bugs: ATLAS-499 > https://issues.apache.org/jira/browse/ATLAS-499 > > > Repository: atlas > > > Description > --- > > ATLAS-499: Automatically update inverse references to prevent repository > corruption from unbalanced references. > > > Diffs > - > > addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14 > addons/hive-bridge/src/test/resources/je.properties PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java > 91f9bd008a6939dfe78656f5c1845637a30ee9eb > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 4f6d0118072380342f5ea302d0aaf7902784849a > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > 4c1f5591f4adead41c3336e64b0bc956836f7edb > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java > 449e066a167ba4546b118a77c8e3de5fd99f077b > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/47658/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
[jira] [Created] (ATLAS-1552) Auto reverse reference update in V2 code path
David Kantor created ATLAS-1552: --- Summary: Auto reverse reference update in V2 code path Key: ATLAS-1552 URL: https://issues.apache.org/jira/browse/ATLAS-1552 Project: Atlas Issue Type: Sub-task Affects Versions: 0.8-incubating Reporter: David Kantor Assignee: David Kantor -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1551) Auto reverse reference update in V1/legacy code path
David Kantor created ATLAS-1551: --- Summary: Auto reverse reference update in V1/legacy code path Key: ATLAS-1551 URL: https://issues.apache.org/jira/browse/ATLAS-1551 Project: Atlas Issue Type: Sub-task Affects Versions: 0.8-incubating Reporter: David Kantor Assignee: David Kantor Attachments: rb56287.patch Review board request for V1 changes: https://reviews.apache.org/r/56287/ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 54899: Atlas-1401: Document setting up development environment with Eclipse
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54899/#review164998 --- Ship it! Ship It! - David Kantor On Feb. 9, 2017, 6:02 p.m., David Radley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54899/ > --- > > (Updated Feb. 9, 2017, 6:02 p.m.) > > > Review request for atlas and Hemanth Yamijala. > > > Bugs: ATLAS-1401 > https://issues.apache.org/jira/browse/ATLAS-1401 > > > Repository: atlas > > > Description > --- > > new file: docs/src/site/twiki/EclipseSetup.twiki > > > Diffs > - > > docs/src/site/twiki/EclipseSetup.twiki PRE-CREATION > docs/src/site/twiki/index.twiki 26157fdd4a9ede7a651d0dc53f51b2e1e535d050 > > Diff: https://reviews.apache.org/r/54899/diff/ > > > Testing > --- > > Ran the Maven build and check the site content is as expected. > > > Thanks, > > David Radley > >
[jira] [Updated] (ATLAS-1539) webapp tests fail when run from the webapp project directory
[
https://issues.apache.org/jira/browse/ATLAS-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1539:
Attachment: rb56463.patch
> webapp tests fail when run from the webapp project directory
>
>
> Key: ATLAS-1539
> URL: https://issues.apache.org/jira/browse/ATLAS-1539
> Project: Atlas
> Issue Type: Bug
> Reporter: David Kantor
> Assignee: David Kantor
> Attachments: rb56463.patch
>
>
> Integration tests in webapp mostly fail when the tests are invoked from the
> webapp project directory. The jetty container used by the tests is
> compromised due to the error shown below, because the
> atlas-application.properties file in the typesystem test jar has properties
> which assume the tests will always be run from the top level e.g.
> {noformat}
> atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
> atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
> {noformat}
> Here is the error which occurs within jetty:
> {noformat}
> 2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
> file,
> filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
> (UserDao:74)
> java.io.FileNotFoundException:
> C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The
> system cannot find the path specified)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
> at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (ATLAS-1539) webapp tests fail when run from the webapp project directory
[
https://issues.apache.org/jira/browse/ATLAS-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1539:
Attachment: (was: rb56463.patch)
> webapp tests fail when run from the webapp project directory
>
>
> Key: ATLAS-1539
> URL: https://issues.apache.org/jira/browse/ATLAS-1539
> Project: Atlas
> Issue Type: Bug
> Reporter: David Kantor
> Assignee: David Kantor
>
> Integration tests in webapp mostly fail when the tests are invoked from the
> webapp project directory. The jetty container used by the tests is
> compromised due to the error shown below, because the
> atlas-application.properties file in the typesystem test jar has properties
> which assume the tests will always be run from the top level e.g.
> {noformat}
> atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
> atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
> {noformat}
> Here is the error which occurs within jetty:
> {noformat}
> 2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
> file,
> filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
> (UserDao:74)
> java.io.FileNotFoundException:
> C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The
> system cannot find the path specified)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
> at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 56287: ATLAS-499 Auto update of reverse reference
> On Feb. 8, 2017, 8:14 p.m., Jeff Hagelberg wrote: > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java, > > line 423 > > <https://reviews.apache.org/r/56287/diff/2/?file=1627188#file1627188line423> > > > > Is it safe to remove this check? I'm not sure it is a safe to assume > > calling ITypedInstance.get(...) will always return something that is > > non-null. This check is not needed as it has already been done above. This was pointed out by a code scan a while back and I had already applied this same change on the fork. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56287/#review164757 ------- On Feb. 9, 2017, 1 a.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56287/ > --- > > (Updated Feb. 9, 2017, 1 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-499 > https://issues.apache.org/jira/browse/ATLAS-499 > > > Repository: atlas > > > Description > --- > > ATLAS-499 auto update of reverse references in V1 API > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > d9c7feb19cc15c16518682e2211b5cfb025d04f5 > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java > f18c308346034a475c65b7103988d542dd8b1796 > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java > PRE-CREATION > > typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java > 5c977c147a63106b810f8855f3e7905b4075e165 > > Diff: https://reviews.apache.org/r/56287/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
Re: Review Request 56287: ATLAS-499 Auto update of reverse reference
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56287/ --- (Updated Feb. 9, 2017, 1 a.m.) Review request for atlas. Changes --- Address review comment Bugs: ATLAS-499 https://issues.apache.org/jira/browse/ATLAS-499 Repository: atlas Description --- ATLAS-499 auto update of reverse references in V1 API Diffs (updated) - repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java d9c7feb19cc15c16518682e2211b5cfb025d04f5 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java f18c308346034a475c65b7103988d542dd8b1796 repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateHardDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateSoftDeleteTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/graph/ReverseReferenceUpdateTestBase.java PRE-CREATION typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java 5c977c147a63106b810f8855f3e7905b4075e165 Diff: https://reviews.apache.org/r/56287/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 9, 2017, 12:47 a.m.) Review request for atlas. Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description (updated) --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files in the typesystem src/test/resources rather than a path relative the top level maven project. Diffs - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 9, 2017, 12:41 a.m.) Review request for atlas. Changes --- Remove whitespace Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files bundled in the typesystem test jar and change bundled atlas-application.properties to specify just the filenames rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 9, 2017, 12:38 a.m.) Review request for atlas. Changes --- Addressed review comment Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files bundled in the typesystem test jar and change bundled atlas-application.properties to specify just the filenames rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 56463: ATLAS-1539 Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-applicatio
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56463/ --- (Updated Feb. 8, 2017, 10:29 p.m.) Review request for atlas. Changes --- Move policy-store.txt and users-credentials.properties to src/test/resources Bugs: ATLAS-1539 https://issues.apache.org/jira/browse/ATLAS-1539 Repository: atlas Description --- Integration tests in projects which use the typesystem test jar (e.g. webapp) were failing when the tests were invoked in the project directory rather than the top level project, because properties for user-credentials.properties and policy store files where using a path relative to the working directory, which amounted to a hard-coded assumption that the tests would only ever be run from the top level. Fix: Attempt to load policy store and user credential files as classloader resource if configured file path does not exist, in the same manner that the atlas-application.properties is located. Use test copies of these files bundled in the typesystem test jar and change bundled atlas-application.properties to specify just the filenames rather than a path relative the top level maven project. Diffs (updated) - authorization/src/main/java/org/apache/atlas/authorize/simple/FileReaderUtil.java 36db7007a14c2afbda7582409e5dfa474fa9a7b6 authorization/src/main/java/org/apache/atlas/authorize/simple/PolicyUtil.java 4f9255a3dee5ac59dcef95e76eae423445be6bdd authorization/src/main/java/org/apache/atlas/authorize/simple/SimpleAtlasAuthorizer.java d6e785366a5fd4dc60496d37b2639ebafa06d087 typesystem/src/test/resources/atlas-application.properties 0e6bc4187ad064820d11da137cee3b8b1857560e typesystem/src/test/resources/policy-store.txt PRE-CREATION typesystem/src/test/resources/users-credentials.properties PRE-CREATION webapp/src/main/java/org/apache/atlas/web/dao/UserDao.java 254d836e5389780b6d10a8d1a664a96b06ad8d1b Diff: https://reviews.apache.org/r/56463/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
[jira] [Updated] (ATLAS-1539) webapp tests fail when run from the webapp project directory
[
https://issues.apache.org/jira/browse/ATLAS-1539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1539:
Attachment: rb56463.patch
Review board request: https://reviews.apache.org/r/56463/
> webapp tests fail when run from the webapp project directory
>
>
> Key: ATLAS-1539
> URL: https://issues.apache.org/jira/browse/ATLAS-1539
> Project: Atlas
> Issue Type: Bug
> Reporter: David Kantor
> Assignee: David Kantor
> Attachments: rb56463.patch
>
>
> Integration tests in webapp mostly fail when the tests are invoked from the
> webapp project directory. The jetty container used by the tests is
> compromised due to the error shown below, because the
> atlas-application.properties file in the typesystem test jar has properties
> which assume the tests will always be run from the top level e.g.
> {noformat}
> atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
> atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
> {noformat}
> Here is the error which occurs within jetty:
> {noformat}
> 2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
> file,
> filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
> (UserDao:74)
> java.io.FileNotFoundException:
> C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The
> system cannot find the path specified)
> at java.io.FileInputStream.open0(Native Method)
> at java.io.FileInputStream.open(FileInputStream.java:195)
> at java.io.FileInputStream.(FileInputStream.java:138)
> at java.io.FileInputStream.(FileInputStream.java:93)
> at
> org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
> at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
Re: Review Request 56417: ATLAS-1535: Some webapp tests are failing due to a stale Titan transaction
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56417/#review164764 --- Ship it! Please start the request description with the motiviation for making these changes i.e. several webapp tests were failing in Titan due to usage of stale transactions left open by previous usage of the thread. webapp/src/main/java/org/apache/atlas/web/filters/StaleTransactionCleanupFilter.java (line 55) <https://reviews.apache.org/r/56417/#comment236526> What happens if there is no open transaction? Is rollback() a no-op or otherwise innocuous? I assume it must be innocuous otherwise there would be other problems. - David Kantor On Feb. 8, 2017, 12:47 a.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56417/ > --- > > (Updated Feb. 8, 2017, 12:47 a.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1535 > https://issues.apache.org/jira/browse/ATLAS-1535 > > > Repository: atlas > > > Description > --- > > I've added a new filter to consistently clean up stale transactions when > processing http requests. I removed the old logic, which was only used by > two service classes. The new filter is used everywhere. > > > Diffs > - > > > webapp/src/main/java/org/apache/atlas/web/filters/StaleTransactionCleanupFilter.java > PRE-CREATION > webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java > d0437fc54087e6803ead83af59b85e9f6df333ad > webapp/src/main/java/org/apache/atlas/web/resources/BaseService.java > fb77b11e04c875a3d443dfb49be50cb818bed441 > > webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java > e1734e465ad553d95735b466f4dfe0fa8ad061b8 > > Diff: https://reviews.apache.org/r/56417/diff/ > > > Testing > --- > > Ran webapp tests. Now, we're only left with the following failures: > > QuickStartIT.runQuickStart:44 » AtlasService Metadata service API > org.apache.a... > QuickStartV2IT.runQuickStart:47 » AtlasService Metadata service API > org.apache... > NotificationHookConsumerIT.testUpdatePartialUpdatingQualifiedName:170 > expected:<0> but was:<1> > > Previously, there were a number of other tests that were failing sporatically > with a NullPointerException coming from Titan. These changes fix that. > > > Thanks, > > Jeff Hagelberg > >
[jira] [Created] (ATLAS-1539) webapp tests fail when run from the webapp project directory
David Kantor created ATLAS-1539:
---
Summary: webapp tests fail when run from the webapp project
directory
Key: ATLAS-1539
URL: https://issues.apache.org/jira/browse/ATLAS-1539
Project: Atlas
Issue Type: Bug
Reporter: David Kantor
Assignee: David Kantor
Integration tests in webapp mostly fail when the tests are invoked from the
webapp project directory. The jetty container used by the tests is compromised
due to the error shown below, because the atlas-application.properties file in
the typesystem test jar has properties which assume the tests will always be
run from the top level e.g.
{noformat}
atlas.auth.policy.file=${sys:user.dir}/distro/src/conf/policy-store.txt
atlas.authentication.method.file.filename=${sys:user.dir}/distro/src/conf/users-credentials.properties
{noformat}
Here is the error which occurs within jetty:
{noformat}
2017-02-07 22:49:33,358 ERROR - [main:] ~ Error while reading user.properties
file,
filepath=C:\source\atlas\webapp/distro/src/conf/users-credentials.properties
(UserDao:74)
java.io.FileNotFoundException:
C:\source\atlas\webapp\distro\src\conf\users-credentials.properties (The system
cannot find the path specified)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at
org.apache.atlas.web.dao.UserDao.loadFileLoginsDetails(UserDao.java:67)
at org.apache.atlas.web.dao.UserDao.init(UserDao.java:53)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (ATLAS-1529) HiveLiteralRewriterTest.setup fails on Windows
[
https://issues.apache.org/jira/browse/ATLAS-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor resolved ATLAS-1529.
-
Resolution: Won't Fix
Just noticed this test was disabled back in Aug 2016 - according to the commit
message, HiveLiteralRewriter is no longer used. Had been running the test on a
fork that was cut prior to the test being disabled.
> HiveLiteralRewriterTest.setup fails on Windows
> --
>
> Key: ATLAS-1529
> URL: https://issues.apache.org/jira/browse/ATLAS-1529
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
> Reporter: David Kantor
>Assignee: David Kantor
>Priority: Minor
>
> {noformat}
> java.lang.RuntimeException: java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:483)
> at org.apache.hadoop.util.Shell.run(Shell.java:456)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
> at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:654)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:629)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:599)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:554)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508)
> at
> org.apache.atlas.hive.bridge.HiveLiteralRewriterTest.setup(HiveLiteralRewriterTest.java:38)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1510) Consolidate/batch calls to GraphBackedTypeStore.findVertex()
[ https://issues.apache.org/jira/browse/ATLAS-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854621#comment-15854621 ] David Kantor commented on ATLAS-1510: - git commit: https://github.com/apache/incubator-atlas/commit/02cf8c4884eb346f7b993bd46d6e789bd73f006d > Consolidate/batch calls to GraphBackedTypeStore.findVertex() > > > Key: ATLAS-1510 > URL: https://issues.apache.org/jira/browse/ATLAS-1510 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > > This showed up as a major hotspot during our profiling of the Atlas > initialization. The idea here is to reduce the number of queries made during > type creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1529) HiveLiteralRewriterTest.setup fails on Windows
David Kantor created ATLAS-1529:
---
Summary: HiveLiteralRewriterTest.setup fails on Windows
Key: ATLAS-1529
URL: https://issues.apache.org/jira/browse/ATLAS-1529
Project: Atlas
Issue Type: Bug
Affects Versions: 0.8-incubating
Reporter: David Kantor
Assignee: David Kantor
Priority: Minor
{noformat}
java.lang.RuntimeException: java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:483)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
at
org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:654)
at
org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:629)
at
org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:599)
at
org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:554)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508)
at
org.apache.atlas.hive.bridge.HiveLiteralRewriterTest.setup(HiveLiteralRewriterTest.java:38)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (ATLAS-1510) Consolidate/batch calls to GraphBackedTypeStore.findVertex()
[ https://issues.apache.org/jira/browse/ATLAS-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854568#comment-15854568 ] David Kantor commented on ATLAS-1510: - [~jnhagelb] Please add a link to the git commit for these changes. > Consolidate/batch calls to GraphBackedTypeStore.findVertex() > > > Key: ATLAS-1510 > URL: https://issues.apache.org/jira/browse/ATLAS-1510 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > > This showed up as a major hotspot during our profiling of the Atlas > initialization. The idea here is to reduce the number of queries made during > type creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 56283: ATLAS-746: After updating a set of entities, response contains only the first entity definition
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/#review164355
---
Fix it, then Ship it!
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java (line 39)
<https://reviews.apache.org/r/56283/#comment236038>
Outdated javadoc, refers to skeleton objects which are not in this
implementation.
repository/src/test/java/org/apache/atlas/TestUtils.java (line 714)
<https://reviews.apache.org/r/56283/#comment236039>
Add javadoc which explains what these methods are doing.
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java (line 1)
<https://reviews.apache.org/r/56283/#comment236040>
Needs Apache header. Remove extra whitespace.
- David Kantor
On Feb. 6, 2017, 3:46 p.m., Jeff Hagelberg wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56283/
> ---
>
> (Updated Feb. 6, 2017, 3:46 p.m.)
>
>
> Review request for atlas and David Kantor.
>
>
> Bugs: ATLAS-746
> https://issues.apache.org/jira/browse/ATLAS-746
>
>
> Repository: atlas
>
>
> Description
> ---
>
> When a an entity is being created that contains nested entities, it is not
> currently possible to determine what guids were assigned to the nested
> entities. This is because the result going back to the caller is simply a
> flat list of the guids of the entities that were created. We get back the
> guids, but it is impossible to know which nested entities they refer to
> without executing an additional query.
>
> This task addresses this issue by adding a "guidAssignments" section to REST
> API output when creating / updating entities. This has a map from the
> "unassigned" guids in the original request to the guids that were assigned
> during the entity creation. This makes it easy to determine what the guid is
> for all the entities that were created, regardlesss of how deeply nested they
> are.
>
> Example:
>
> {
> Â Â Â "entitiesMutated":{
>  …
> Â Â Â },
> Â Â Â "guidAssignments":{
> Â Â "-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
> Â Â "-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
> Â Â "-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
> Â Â "-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
> Â Â "-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
> Â Â "-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
> Â Â "-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
> Â Â "-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
> Â Â "-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
> Â Â "-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
> Â Â "-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
> Â Â "-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
> Â Â "-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
> Â Â "-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
> Â Â "-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
> Â Â "-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
> Â Â "-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
> Â Â "-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
> Â Â "-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
> Â Â "-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
> Â Â "-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
> Â Â Â }
> }
>
>
> I also fixed an issue preventing nested entities from being created. The
> issue there was that the mapping composite attribute representation in the
> old and new format was inconsistent. I've changed the code so that an
> attribute is treated as composite if it is either a "mappedFromRef" attribute
> or it has a foreign key with "update"as the onDelete action. There were two
> competing ways of determining this.
>
> One quick note about
> EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on
> the surface,
Re: Review Request 56126: ATLAS-1510: Consolidate/batch calls to GraphBackedTypeStore.findVertex()
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56126/#review163991 --- Fix it, then Ship it! repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java (line 95) <https://reviews.apache.org/r/56126/#comment235497> typo - David Kantor On Jan. 31, 2017, 8:22 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56126/ > --- > > (Updated Jan. 31, 2017, 8:22 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1510 > https://issues.apache.org/jira/browse/ATLAS-1510 > > > Repository: atlas > > > Description > --- > > These changes optimize GraphBackedTypeStore.store() so that it does all of > the required vertex lookup up front, using one query. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > ce8d4c75610a1337d72134bf245c72e375ad4984 > > repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java > c1c2a94e4367d4c4963a6d2807b1fe1d818102f7 > > repository/src/main/java/org/apache/atlas/repository/typestore/TypePersistenceVisitor.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/typestore/TypeVertexFinder.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/typestore/TypeVertexInfo.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/typestore/TypeVisitor.java > PRE-CREATION > > repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java > 6b83aa3222c15ebad92399a6668ee5a802b10802 > > Diff: https://reviews.apache.org/r/56126/diff/ > > > Testing > --- > > Ran entire Atlas build, no regressions found. > > > Thanks, > > Jeff Hagelberg > >
Re: Review Request 54858: Atlas-1178: Use .gitignore so git does not see binary files as changed
> On Jan. 30, 2017, 9:04 p.m., David Kantor wrote: > > .gitignore, line 1 > > <https://reviews.apache.org/r/54858/diff/3/?file=1604866#file1604866line1> > > > > Please set the bugs field to ATLAS-1378 and fix the summary to mention > > ATLAS-1378 rather than 1178. Please set the bugs field to ATLAS-1378 and fix the summary to mention ATLAS-1378 rather than 1178. Also, please close this request as Submitted. To do that, go to My Dashboard, click the check box next to this request, and then click Close Submitted. Thanks... - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54858/#review163570 --- On Jan. 13, 2017, 4:35 p.m., David Radley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54858/ > --- > > (Updated Jan. 13, 2017, 4:35 p.m.) > > > Review request for atlas and David Kantor. > > > Repository: atlas > > > Description > --- > > Added .gitignore files so git does not see binary files as changed. Atlas > type system project has a strange .gitignore that I have not touched. I > notice it contains ignores for Eclipse files, but the way we run it the > eclipse files are not picked up by git. > > > Diffs > - > > .gitignore cf2ef7445407f846329b0849d23dcb0aecc4d242 > > Diff: https://reviews.apache.org/r/54858/diff/ > > > Testing > --- > > git status shows no untracked changes after a compile. > > > Thanks, > > David Radley > >
Re: Review Request 54759: ATLAS-1379 Avoid object query overhead when report query selects class type alias
> On Jan. 5, 2017, 2:19 p.m., David Kantor wrote: > > Hi Neeru, I've committed these changes and resolved ATLAS-1379. Please > > close this review request as submitted. Thanks... As requested on 1/5, please close this request as submitted, thanks. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54759/#review160585 --- On Jan. 3, 2017, 8:21 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54759/ > --- > > (Updated Jan. 3, 2017, 8:21 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1379 > https://issues.apache.org/jira/browse/ATLAS-1379 > > > Repository: atlas > > > Description > --- > > OMS-743 when an object instance is selected in select query then only fetch > the id (guid, typename, version and state) for performance reasons > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java > fb027efc1ad9a7b2c562fc6670a8f9647b3b54f0 > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > aad056f5247cd5ac0e5c84d2f1fa9c4bb5458fb3 > > repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java > f2a1fe975bc720e3003389806f1e32d3edf74713 > > repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java > a2a9ab6dafcefaa29110051c35d3b6762017a28f > > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java > 51e9ab1fb6bfe7e854bcf40bfc3d92136de4c60f > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java > 2c12bc3e82109819759c3c54ba3a786fee656594 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > f6b3fc5ffb6dd76271ccf8217b587b52b98cf78b > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 7eacc5dbf186885a3e51294b8717bc9c05bd0b66 > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > 19ca328a7f4240ea62fdd171f23c6c5c14d25af5 > > repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala > eb6d119b2eca699d67fe071fe2659de2fddb9285 > repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala > 9e336cf2abb90141a16220cae078952cdc410475 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > aaa8fa0d594e97fabfa51c5c76c5aab879bf8e7f > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java > d73a7b32add5d9e4eb24afefed042cf894964992 > > Diff: https://reviews.apache.org/r/54759/diff/ > > > Testing > --- > > All Unit tests executed successfully. > > Added new test cases in GraphBackedDiscoveryTest to valiadte the fields > coming back with id type report queries. > > > Thanks, > > Neeru Gupta > >
Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56102/#review163974 --- Ship it! Ship It! - David Kantor On Feb. 1, 2017, 9:41 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56102/ > --- > > (Updated Feb. 1, 2017, 9:41 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1388 > https://issues.apache.org/jira/browse/ATLAS-1388 > > > Repository: atlas > > > Description > --- > > Cache created entities in RequestContext when they are created. > Update/refactor DefaultMetadataService.loadEntities() to use the cached > created entities. > Use the cache in the following places: > FullTextMapper > DefaultMetadataService.onEntitiesAdded - check cache before calling > DefaultMetadataService.loadEntities > DefaultMetadataService.onEntitiesUpdated - check cache before calling > DefaultMetadataService.loadEntities > EntityResource.getEntityDefinition - check cache before calling > DefaultMetadataService.getEntityDefinition > EntityResource.getResponse - check cache before calling > DefaultMetadataService.getEntityDefinition > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > f4d8f00da863deeb98209aec2dfc4ccb22734cfa > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java > 5be8d0bc349294dc4340ffe5831bfb1265825ed8 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > 6608551be44033f07ddbd2ac3f3764d1ac3b0f22 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java > 38a553abf0151221c5d802e9ce1977a693d1a698 > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > bae8b2ac0cae6bf6392f91f24030d81089ae660b > repository/src/test/java/org/apache/atlas/TestUtils.java > cda9eac38861fd55c1494d6d90b5e750e6133545 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > d447c2d90fec6c03c7fecb8d8ef57a6ea072d35a > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java > 9e850a9dc0ac122cf3fcdbf7df72a93643abc98d > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java > 7444bf38834f59454d67cf763c932d665d7ef31c > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java > c902f8126f6b516e84ab9d89b0ec80ac8ed819fd > > repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java > 03ef4fe02f9b4eba273297ea954f00c9181f1f02 > server-api/src/main/java/org/apache/atlas/RequestContext.java > 651a71dc238a3adaac9504a77ba439785fa07ca8 > > Diff: https://reviews.apache.org/r/56102/diff/ > > > Testing > --- > > Ran full build, no regresssions found. > > > Thanks, > > Jeff Hagelberg > >
[jira] [Commented] (ATLAS-1514) Duplicates in class array attribute are not removed during delete
[ https://issues.apache.org/jira/browse/ATLAS-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849081#comment-15849081 ] David Kantor commented on ATLAS-1514: - Committed to master - https://github.com/apache/incubator-atlas/commit/a810f15648e764d1e15f1e810718da8eba7758f6 > Duplicates in class array attribute are not removed during delete > - > > Key: ATLAS-1514 > URL: https://issues.apache.org/jira/browse/ATLAS-1514 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: David Kantor > > Duplicates entries in a class array attribute are not removed when the target > entity is deleted, leaving a dangling reference. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated
> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote: > > The cache size can grow large if the number of registered entities is high. > > It would be good to have a cache eviction policy just like it was done for > > DSL Query caching in ATLAS-1387 > > Jeff Hagelberg wrote: > We could do that. It seems like overkill though. The big difference is > that RequestContext is thread local, and a new RequestContext is created > every time a http request comes in. Once the http request finishes, > RequestContext.clear() is called, and the cache goes out of scope and will be > garbage collected. (See AuditFilter for details). The cache is limited to > the instances used for processsing the request. In the case of the compiled > query cache, there is one effectively static instance of the cache that is > used for all requests. Without the eviction policy, that would continue > growing until the JVM runs out of memory. > > Jeff Hagelberg wrote: > Is there a specific use case you are concerned about? For example, if > someone runs a DSL query that needs to return 100,000 entities, those > entities would be cached. However, even without the cache we would still > need to have the 100,000 entities in memory so they can be serialized to json > and sent back to the client. I'm not sure that the cache increases the > amount of memory required by much. What it does do, though, is keep the > instances that are created in the java heap for a longer period of time. There no need for a cache eviction policy as this cache is only active for the duration of the request. Perhaps to address the comment, the cache could be cleared upon completion of the request to make the entries available for garbage collection sooner. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56102/#review163800 --- On Jan. 30, 2017, 10:42 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56102/ > ------- > > (Updated Jan. 30, 2017, 10:42 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1388 > https://issues.apache.org/jira/browse/ATLAS-1388 > > > Repository: atlas > > > Description > --- > > Cache created entities in RequestContext when they are created. > Update/refactor DefaultMetadataService.loadEntities() to use the cached > created entities. > Use the cache in the following places: > FullTextMapper > DefaultMetadataService.onEntitiesAdded - check cache before calling > DefaultMetadataService.loadEntities > DefaultMetadataService.onEntitiesUpdated - check cache before calling > DefaultMetadataService.loadEntities > EntityResource.getEntityDefinition - check cache before calling > DefaultMetadataService.getEntityDefinition > EntityResource.getResponse - check cache before calling > DefaultMetadataService.getEntityDefinition > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > f4d8f00da863deeb98209aec2dfc4ccb22734cfa > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java > 5be8d0bc349294dc4340ffe5831bfb1265825ed8 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > be02891cad3020e50d5244fd2aaf3c4af3a5954f > > repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java > 7b2b753b930467e015d364819c5baa983fce0cf8 > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > bae8b2ac0cae6bf6392f91f24030d81089ae660b > repository/src/test/java/org/apache/atlas/TestUtils.java > cda9eac38861fd55c1494d6d90b5e750e6133545 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > f2ca6a8fb1573755d16497236358358a07881e9c > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java > 9e850a9dc0ac122cf3fcdbf7df72a93643abc98d > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java > 7444bf38834f59454d67cf763c932d665d7ef31c > > repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java > c902f8126f6b516e84ab9d89b0ec80ac8ed819fd > > repository/src/test/java/org
[jira] [Commented] (ATLAS-1514) Duplicates in class array attribute are not removed during delete
[ https://issues.apache.org/jira/browse/ATLAS-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848466#comment-15848466 ] David Kantor commented on ATLAS-1514: - In progress > Duplicates in class array attribute are not removed during delete > - > > Key: ATLAS-1514 > URL: https://issues.apache.org/jira/browse/ATLAS-1514 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Reporter: David Kantor > > Duplicates entries in a class array attribute are not removed when the target > entity is deleted, leaving a dangling reference. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (ATLAS-1514) Duplicates in class array attribute are not removed during delete
David Kantor created ATLAS-1514: --- Summary: Duplicates in class array attribute are not removed during delete Key: ATLAS-1514 URL: https://issues.apache.org/jira/browse/ATLAS-1514 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: David Kantor Duplicates entries in a class array attribute are not removed when the target entity is deleted, leaving a dangling reference. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
Re: Review Request 54680: ATLAS-1369 - Optimize gremlin queries generated by DSL translator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54680/#review163667 --- Fix it, then Ship it! Fix it then ship it common/src/main/java/org/apache/atlas/groovy/ClosureExpression.java (line 35) <https://reviews.apache.org/r/54680/#comment235141> Consider renaming to VariableDeclaration, no need to abbreviate the name. common/src/main/java/org/apache/atlas/groovy/CodeBlockExpression.java (line 25) <https://reviews.apache.org/r/54680/#comment235140> typo repository/src/main/java/org/apache/atlas/gremlin/Gremlin2ExpressionFactory.java (line 185) <https://reviews.apache.org/r/54680/#comment235155> remove commented-out code repository/src/main/java/org/apache/atlas/gremlin/Gremlin3ExpressionFactory.java (lines 291 - 292) <https://reviews.apache.org/r/54680/#comment235157> Rename to startIndex and endIndex repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpandOrsOptimization.java (line 43) <https://reviews.apache.org/r/54680/#comment235158> Is it Gremlin that does not use the indices, or is it Titan? I thought that was a Titan bug. repository/src/main/java/org/apache/atlas/gremlin/optimizer/GremlinQueryOptimizer.java (lines 34 - 35) <https://reviews.apache.org/r/54680/#comment235161> It's the GremlinOptimization implementations that apply the specific optimizations. This javadoc implies that GremlinQueryOptimizer is doing it. Perhaps it makes sense to have the details of each optimization documented in the implementation classes, and change this javadoc to be more general. repository/src/main/java/org/apache/atlas/gremlin/optimizer/OptimizationContext.java (lines 40 - 42) <https://reviews.apache.org/r/54680/#comment235162> Remove empty comment repository/src/main/java/org/apache/atlas/gremlin/optimizer/PathExpressionFinder.java (line 25) <https://reviews.apache.org/r/54680/#comment235163> Incorrect javadoc, should mention path expressions not aliases. repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java (lines 157 - 160) <https://reviews.apache.org/r/54680/#comment235166> I think is comment is obsolete - a holdover from the original superTypes optimization - and should be removed. - David Kantor On Jan. 31, 2017, 2:11 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54680/ > --- > > (Updated Jan. 31, 2017, 2:11 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1369 > https://issues.apache.org/jira/browse/ATLAS-1369 > > > Repository: atlas > > > Description > --- > > The way the queries are generated now, in many cases the indices are not used > when executing queries. The main problem is that gremlin cannot currently use > "has" expressions within an "and" or an "or". This JIRA introduces a > GremlinQueryOptimizer that attempts to extract "has" expressions out of "and" > expressions and turn "or" expressions into unions. > > > Major changes: > > * Updated version of Guava used by Atlas, so that could use > Lists.cartesianProduct > * New package - org.apache.atlas.gremlin.optimizer - contains > optimization framework > * Refactoring of Groovy Expression classes > * new - AbstractFunctionExpression - super class of anything with a > caller > * added 4 new methods to GroovyExpression > * now, each GroovyExpression is a TraversalStepType associated with it > These are used by the optimization framework > > * Added feature toggle : atlas.query.gremlinOptimizerEnabled. The feature > turns optimization on/off. It is provided as a fallback in the event that > issues are found with the optimizer. > > * Most of the new logic is in GremlinQueryOptimizer, > ExpandAndsOptimization, and ExpandOrsOptimization > * > Note - with these changes we saw a very significant improvement when using > IBM Graph as the Atlas Backend (which uses Titan 1 under the covers). > > Please review these changes. Silence implies consent. > > > Diffs > - > > > common/src/main/java/org/apache/atlas/groovy/AbstractFunctionExpression.java > PRE-CREATION > common/src/main/java/org/apache/atlas/groovy/AbstractGroovyExpression.java > 49eaae87e593d4e350ecb9a51a83e967d95b514b > common/src/main/java/org/apache/atlas/groovy/ArithmeticExpression.ja
Re: Review Request 54858: Atlas-1178: Use .gitignore so git does not see binary files as changed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54858/#review163571 --- Ship it! Ship It! - David Kantor On Jan. 13, 2017, 4:35 p.m., David Radley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54858/ > --- > > (Updated Jan. 13, 2017, 4:35 p.m.) > > > Review request for atlas and David Kantor. > > > Repository: atlas > > > Description > --- > > Added .gitignore files so git does not see binary files as changed. Atlas > type system project has a strange .gitignore that I have not touched. I > notice it contains ignores for Eclipse files, but the way we run it the > eclipse files are not picked up by git. > > > Diffs > - > > .gitignore cf2ef7445407f846329b0849d23dcb0aecc4d242 > > Diff: https://reviews.apache.org/r/54858/diff/ > > > Testing > --- > > git status shows no untracked changes after a compile. > > > Thanks, > > David Radley > >
Re: Review Request 54858: Atlas-1178: Use .gitignore so git does not see binary files as changed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54858/#review163570 --- .gitignore (line 1) <https://reviews.apache.org/r/54858/#comment235024> Please set the bugs field to ATLAS-1378 and fix the summary to mention ATLAS-1378 rather than 1178. - David Kantor On Jan. 13, 2017, 4:35 p.m., David Radley wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54858/ > --- > > (Updated Jan. 13, 2017, 4:35 p.m.) > > > Review request for atlas and David Kantor. > > > Repository: atlas > > > Description > --- > > Added .gitignore files so git does not see binary files as changed. Atlas > type system project has a strange .gitignore that I have not touched. I > notice it contains ignores for Eclipse files, but the way we run it the > eclipse files are not picked up by git. > > > Diffs > - > > .gitignore cf2ef7445407f846329b0849d23dcb0aecc4d242 > > Diff: https://reviews.apache.org/r/54858/diff/ > > > Testing > --- > > git status shows no untracked changes after a compile. > > > Thanks, > > David Radley > >
Re: Review Request 54759: ATLAS-1379 Avoid object query overhead when report query selects class type alias
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54759/#review160451 --- This change has been available for review since Dec 16. It will be committed tomorrow (Wed Jan 4 2016) unless there are further comments on review board. - David Kantor On Jan. 3, 2017, 8:21 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54759/ > --- > > (Updated Jan. 3, 2017, 8:21 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1379 > https://issues.apache.org/jira/browse/ATLAS-1379 > > > Repository: atlas > > > Description > --- > > OMS-743 when an object instance is selected in select query then only fetch > the id (guid, typename, version and state) for performance reasons > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java > fb027efc1ad9a7b2c562fc6670a8f9647b3b54f0 > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > aad056f5247cd5ac0e5c84d2f1fa9c4bb5458fb3 > > repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java > f2a1fe975bc720e3003389806f1e32d3edf74713 > > repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java > a2a9ab6dafcefaa29110051c35d3b6762017a28f > > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java > 51e9ab1fb6bfe7e854bcf40bfc3d92136de4c60f > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java > 2c12bc3e82109819759c3c54ba3a786fee656594 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > f6b3fc5ffb6dd76271ccf8217b587b52b98cf78b > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 7eacc5dbf186885a3e51294b8717bc9c05bd0b66 > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > 19ca328a7f4240ea62fdd171f23c6c5c14d25af5 > > repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala > eb6d119b2eca699d67fe071fe2659de2fddb9285 > repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala > 9e336cf2abb90141a16220cae078952cdc410475 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > aaa8fa0d594e97fabfa51c5c76c5aab879bf8e7f > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java > d73a7b32add5d9e4eb24afefed042cf894964992 > > Diff: https://reviews.apache.org/r/54759/diff/ > > > Testing > --- > > All Unit tests executed successfully. > > Added new test cases in GraphBackedDiscoveryTest to valiadte the fields > coming back with id type report queries. > > > Thanks, > > Neeru Gupta > >
[jira] [Updated] (ATLAS-1386) Avoid unnecessary type lookups during query parsing
[ https://issues.apache.org/jira/browse/ATLAS-1386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1386: Assignee: Jeffrey Hagelberg > Avoid unnecessary type lookups during query parsing > --- > > Key: ATLAS-1386 > URL: https://issues.apache.org/jira/browse/ATLAS-1386 > Project: Atlas > Issue Type: Improvement > Reporter: David Kantor >Assignee: Jeffrey Hagelberg > > During query parsing, there are unnecessary types lookups happening. For > example, in the DSL query hive_table where name = \"bar-999\" select name, > name is parsed into an identifier, and it was doing a type lookup to see if a > trait or class called 'name' exists. In cases like this, the operands are > fields, not types, and so a type lookup is not necessary and arguably > incorrect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1387) Cache of compiled DSL queries
[ https://issues.apache.org/jira/browse/ATLAS-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1387: Assignee: Jeffrey Hagelberg > Cache of compiled DSL queries > - > > Key: ATLAS-1387 > URL: https://issues.apache.org/jira/browse/ATLAS-1387 > Project: Atlas > Issue Type: Improvement > Reporter: David Kantor >Assignee: Jeffrey Hagelberg > > Add a cache of compiled DSL queries to avoid unnecessary overhead of > re-compiling/transaction the same query multiple times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-1385) Add configuration property to disable full text mapper
[ https://issues.apache.org/jira/browse/ATLAS-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1385: Assignee: Jeffrey Hagelberg > Add configuration property to disable full text mapper > -- > > Key: ATLAS-1385 > URL: https://issues.apache.org/jira/browse/ATLAS-1385 > Project: Atlas > Issue Type: New Feature > Reporter: David Kantor >Assignee: Jeffrey Hagelberg > > Add configuration property to disable full text mapper. Populating the full > text fields adds considerable overhead, and this allows applications to avoid > this overhead if they do not need full text search capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1390) IBM Graph implementation of graph DB abstraction
David Kantor created ATLAS-1390: --- Summary: IBM Graph implementation of graph DB abstraction Key: ATLAS-1390 URL: https://issues.apache.org/jira/browse/ATLAS-1390 Project: Atlas Issue Type: New Feature Reporter: David Kantor Assignee: Jeffrey Hagelberg Support running Atlas against IBM Graph as the graph backend via IBM Graph implementation of the graph DB abstraction. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1389) Ping REST API entrypoint
David Kantor created ATLAS-1389: --- Summary: Ping REST API entrypoint Key: ATLAS-1389 URL: https://issues.apache.org/jira/browse/ATLAS-1389 Project: Atlas Issue Type: New Feature Reporter: David Kantor Assignee: Jeffrey Hagelberg Provide a lightweight ping API to allow clients to check status of Atlas server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1388) Cache entities that are created/updated
David Kantor created ATLAS-1388: --- Summary: Cache entities that are created/updated Key: ATLAS-1388 URL: https://issues.apache.org/jira/browse/ATLAS-1388 Project: Atlas Issue Type: Improvement Reporter: David Kantor Assignee: Jeffrey Hagelberg - Cache created entities in RequestContext when they are created. - Update/refactor DefaultMetadataService.loadEntities() to use the cached created entities. - Use the cache in the following places: ** FullTextMapper ** DefaultMetadataService.onEntitiesAdded - check cache before calling DefaultMetadataService.loadEntities ** DefaultMetadataService.onEntitiesUpdated - check cache before calling DefaultMetadataService.loadEntities ** EntityResource.getEntityDefinition - check cache before calling DefaultMetadataService.getEntityDefinition ** EntityResource.getResponse - check cache before calling DefaultMetadataService.getEntityDefinition -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1387) Cache of compiled DSL queries
David Kantor created ATLAS-1387: --- Summary: Cache of compiled DSL queries Key: ATLAS-1387 URL: https://issues.apache.org/jira/browse/ATLAS-1387 Project: Atlas Issue Type: Improvement Reporter: David Kantor Add a cache of compiled DSL queries to avoid unnecessary overhead of re-compiling/transaction the same query multiple times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1386) Avoid unnecessary type lookups during query parsing
David Kantor created ATLAS-1386: --- Summary: Avoid unnecessary type lookups during query parsing Key: ATLAS-1386 URL: https://issues.apache.org/jira/browse/ATLAS-1386 Project: Atlas Issue Type: Improvement Reporter: David Kantor During query parsing, there are unnecessary types lookups happening. For example, in the DSL query hive_table where name = \"bar-999\" select name, name is parsed into an identifier, and it was doing a type lookup to see if a trait or class called 'name' exists. In cases like this, the operands are fields, not types, and so a type lookup is not necessary and arguably incorrect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-1385) Add configuration property to disable full text mapper
David Kantor created ATLAS-1385: --- Summary: Add configuration property to disable full text mapper Key: ATLAS-1385 URL: https://issues.apache.org/jira/browse/ATLAS-1385 Project: Atlas Issue Type: New Feature Reporter: David Kantor Add configuration property to disable full text mapper. Populating the full text fields adds considerable overhead, and this allows applications to avoid this overhead if they do not need full text search capability. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 54759: ATLAS-1379 Avoid object query overhead when report query selects class type alias
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54759/#review159368 --- Ship it! Ship It! - David Kantor On Dec. 15, 2016, 5:20 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54759/ > --- > > (Updated Dec. 15, 2016, 5:20 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-1379 > https://issues.apache.org/jira/browse/ATLAS-1379 > > > Repository: atlas > > > Description > --- > > OMS-743 when an object instance is selected in select query then only fetch > the id (guid, typename, version and state) for performance reasons > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java > fcf120ab3db933a4ecd10f9427c9fc458ea10627 > > repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java > aad056f5247cd5ac0e5c84d2f1fa9c4bb5458fb3 > > repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java > c0cc25ca7abbda1adca99c6ca10fa473c4b76913 > > repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java > a2a9ab6dafcefaa29110051c35d3b6762017a28f > > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java > ae1ec45a521660648028add0d63f0f6cadb4a495 > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java > 2c12bc3e82109819759c3c54ba3a786fee656594 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > 1a3faf778f01582e476792780a9dbaa8a4010d29 > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > cb54c3e66a7f9910ad45d76691397817134b0c22 > > repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java > 78e276e5972f6bba68198b3ae5a3d88676cf7585 > > repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala > eb6d119b2eca699d67fe071fe2659de2fddb9285 > repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala > 9e336cf2abb90141a16220cae078952cdc410475 > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > fba6d195597fbc956dc615721bd92c7074f5f608 > typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java > d73a7b32add5d9e4eb24afefed042cf894964992 > > Diff: https://reviews.apache.org/r/54759/diff/ > > > Testing > --- > > All Unit tests executed successfully. > > Added new test cases in GraphBackedDiscoveryTest to valiadte the fields > coming back with id type report queries. > > > Thanks, > > Neeru Gupta > >
Re: Review Request 54430: Atlas-1186: Add glossary Categories
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54430/#review159055
---
General comments:
* Indentation is misaligned in several files.
* Remove all extra whitespace denoted by red blocks
* Remove any commented-out code
* Always use spaces for indentation, no tabs please
As for the functionality itself - it seems fine, but I am frankly not
knowledgable in this area and have not had any experience working with the
functionality in the catalog project I would defer to others in the community
on this, and also to IBMers like Mandy Chessell, Kevin Shank, Nigel Jones, and
Stephanie Hazelwood, as they have much deeper understanding of this domain than
I do.
catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java (line 114)
<https://reviews.apache.org/r/54430/#comment229988>
Remove all extra whitespace denoted by red blocks. This comments applies to
all files in this change set.
catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java (line 124)
<https://reviews.apache.org/r/54430/#comment229991>
typo
catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java (line 129)
<https://reviews.apache.org/r/54430/#comment229992>
This is exposing the implementation detail that this data is stored in a
graph. Firstly, I think users should insulated from this. Secondly, what is
the use case?
catalog/src/main/java/org/apache/atlas/catalog/AtlasTypeSystem.java (line 145)
<https://reviews.apache.org/r/54430/#comment229990>
Why does take a request and other methods take a String guid parameter?
Also, I would strongly recommend changing this to support deleting multiple
entities with one call i.e. deleteEntitiesById which takes a List guids
arg.
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java (line 142)
<https://reviews.apache.org/r/54430/#comment229993>
See earlier comment about supporting deletion of multiple entities.
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java (line 145)
<https://reviews.apache.org/r/54430/#comment229994>
This should delegate to MetadataService.deleteEntities
catalog/src/main/java/org/apache/atlas/catalog/EntityResourceProvider.java
(lines 45 - 47)
<https://reviews.apache.org/r/54430/#comment229995>
Refactor this code into a method, as I see it is duplicated in
getResources()
catalog/src/main/java/org/apache/atlas/catalog/TaxonomyResourceProvider.java
(lines 105 - 124)
<https://reviews.apache.org/r/54430/#comment229996>
Add comments to describe what is going on in this algorithm
catalog/src/main/java/org/apache/atlas/catalog/definition/GlossaryResourceDefinition.java
(line 36)
<https://reviews.apache.org/r/54430/#comment229997>
Why is this class package private?
catalog/src/test/java/org/apache/atlas/catalog/GlossaryCategoryResourceProviderTest.java
(line 505)
<https://reviews.apache.org/r/54430/#comment229998>
Remove commented code
catalog/src/test/java/org/apache/atlas/catalog/GlossaryCategoryResourceProviderTest.java
(lines 1004 - 1005)
<https://reviews.apache.org/r/54430/#comment22>
Remove commented code
catalog/src/test/java/org/apache/atlas/catalog/definition/GlossaryCategoryResourceDefintionTest.java
(line 39)
<https://reviews.apache.org/r/54430/#comment23>
Incorrect javadoc
catalog/src/test/java/org/apache/atlas/catalog/definition/GlossaryCategoryResourceDefintionTest.java
(line 41)
<https://reviews.apache.org/r/54430/#comment230001>
Typo in class name, should be GlossaryCategoryResourceDefinitionTest
repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java
(line 255)
<https://reviews.apache.org/r/54430/#comment230002>
All of these info-level debug logs should be removed
repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java
(line 410)
<https://reviews.apache.org/r/54430/#comment230003>
remove this or change the level
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
(line 319)
<https://reviews.apache.org/r/54430/#comment230004>
looks like typo, i.e there is no space between param name and comment
comment
- David Kantor
On Dec. 7, 2016, 2:42 p.m., David Radley wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54430/
> ---
>
> (Updated Dec. 7, 2016, 2:42 p.m.)
>
>
> Review request for atlas, DaveEL DaveEL, Shwetha GS, and Hemanth Yamijala.
>
>
> Repository: atlas
>
>
> Description
> ---
>
> I am loo
[jira] [Commented] (ATLAS-1284) Fix order by clause to be able to order on super type attributes and with alias.
[ https://issues.apache.org/jira/browse/ATLAS-1284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15726771#comment-15726771 ] David Kantor commented on ATLAS-1284: - [~guptaneeru] Is this defect addressed by your changes for ATLAS-737? If yes, please resolve this as Fixed and indicate as such in a comment. If not, please update the [review board request for ATLAS-737|https://reviews.apache.org/r/53653] to replace ATLAS-1284 with ATLAS-737 in the description, and remove ATLAS-1284 from the list of bugs. > Fix order by clause to be able to order on super type attributes and with > alias. > > > Key: ATLAS-1284 > URL: https://issues.apache.org/jira/browse/ATLAS-1284 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Neeru Gupta >Assignee: Neeru Gupta > > Noticed issues with order by to work with alias and while sorting on super > type attributes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Review Request 53653: ATLAS-1284 Fixed issues with order by clause while sorting with alias and on super type attributes
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/53653/#review157795 --- Ship it! - David Kantor On Dec. 2, 2016, 4:52 p.m., Neeru Gupta wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/53653/ > --- > > (Updated Dec. 2, 2016, 4:52 p.m.) > > > Review request for atlas and Jeff Hagelberg. > > > Bugs: Atlas-1284 and Atlas-737 > https://issues.apache.org/jira/browse/Atlas-1284 > https://issues.apache.org/jira/browse/Atlas-737 > > > Repository: atlas > > > Description > --- > > Aggregate methods and order by fixes. > > > Diffs > - > > docs/src/site/twiki/Search.twiki 5b9431a4912315e6114cfe5333a6a6d89e87d4d1 > > repository/src/main/java/org/apache/atlas/gremlin/Gremlin2ExpressionFactory.java > a901b5127b3dcd2939fec2c642475d7aad869d8e > > repository/src/main/java/org/apache/atlas/gremlin/Gremlin3ExpressionFactory.java > 32189bc6dab4cc25748e7b6136dc0a89476012c7 > > repository/src/main/java/org/apache/atlas/gremlin/GremlinExpressionFactory.java > 9a36e780850d0c349343d6a751c5e7f181c86198 > repository/src/main/scala/org/apache/atlas/query/Expressions.scala > 297aa2b8f917d5ae25da6a69901af60542082642 > repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala > 6803036df7ef7b7c7c08579f1f7cbd4c731aec46 > repository/src/main/scala/org/apache/atlas/query/GremlinQuery.scala > ee58eee7b4b2c41cc8f6dc1e4952b83ed0c833df > repository/src/main/scala/org/apache/atlas/query/QueryParser.scala > 7074d74770a5098ca82852919f236fd66c6d4a84 > repository/src/main/scala/org/apache/atlas/query/Resolver.scala > cff92afe98a6215499a70a9ef00a937804abe73e > repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java > 8851b793b0ca032efb57bd76ec9198822e1520c2 > repository/src/test/java/org/apache/atlas/TestUtils.java > 88edf8f03e356dc4850c16d99f3deb3370d8546d > > repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java > 4ac001e7fab7bdea744ec275de39ea1fced77903 > > Diff: https://reviews.apache.org/r/53653/diff/ > > > Testing > --- > > Order by fixes to add ability to orderby on supertypes attributes and with > alias. > > Added new aggregate methods support like group by and count, max, min and sum > methods. > > Executed all existing order by tests successfully. Added new tests to order > by with alias > > > Thanks, > > Neeru Gupta > >
[jira] [Commented] (ATLAS-582) Move Atlas UI to use backboneJS
[ https://issues.apache.org/jira/browse/ATLAS-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15616617#comment-15616617 ] David Kantor commented on ATLAS-582: [~kevalbhatt] What specific version of the animate.css package was dashboardv2/public/animate.min.css obtained from? Please advise, as that info is needed to fully clear the licensing of this module. I tried to match it up with several versions listed in the history on https://github.com/daneden/animate.css/blob/master/animate.min.css, but could not find an exact match. Thanks for your help. > Move Atlas UI to use backboneJS > --- > > Key: ATLAS-582 > URL: https://issues.apache.org/jira/browse/ATLAS-582 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Keval Bhatt > Fix For: 0.7-incubating > > Attachments: ATLAS-582.5.patch, ATLAS-582.6.patch, ATLAS-582.7.patch, > ATLAS-582.patch > > > Desire is to move Atlas UI implementation to backboneJS for greater > flexibility in the implementation of the UI. Specifically, backbone provides > a simple way to represent models and collections as UI components. All > components are also "skinnable" with CSS, and will render consistently across > browsers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-1195) Clean up DSL Translation
[ https://issues.apache.org/jira/browse/ATLAS-1195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15612808#comment-15612808 ] David Kantor edited comment on ATLAS-1195 at 10/27/16 7:00 PM: --- +1 Review approved by [~suma.shivaprasad] Committed to trunk - thanks [~jnhagelb] was (Author: dkantor): Committed to trunk - thanks [~jnhagelb] > Clean up DSL Translation > > > Key: ATLAS-1195 > URL: https://issues.apache.org/jira/browse/ATLAS-1195 > Project: Atlas > Issue Type: Sub-task >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > Fix For: 0.8-incubating > > Attachments: rb53133.patch > > > Right now, the DSL translation logic is full if statements like if(TP2) then > do this otherwise do that. This needs to be cleaned up. We need classes > that represent each gremlin construct and subclasses for TP2 / TP3 when > appropriate. We should first look and see if the existing Expression classes > can be leveraged for this. We should also try to make this general enough so > that whatever logic we put in place can also be leveraged by the catalog > project to generate TP2/TP3 gremlin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-693) Add titan 0.5.4 implemenation of graph database abstraction layer
[ https://issues.apache.org/jira/browse/ATLAS-693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-693: --- Fix Version/s: 0.8-incubating > Add titan 0.5.4 implemenation of graph database abstraction layer > - > > Key: ATLAS-693 > URL: https://issues.apache.org/jira/browse/ATLAS-693 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > Fix For: trunk, 0.8-incubating > > Attachments: rb47466.patch > > > In this task, we put in place the titan 0.5.4 implementation of the graph > database abstraction layer. As part of this effort, the existing "titan" > maven project will be merged into this new project, since it is specific to > titan 0.5.4. The maven projects in Atlas that depend on the "titan" will be > changed to depend on the new project. Direct dependencies on titan 0.5.4 > will be removed from all of the atlas projects. These projects will be > changed to depend on this new project instead. They will transitively obtain > the dependencies on titan 0.5.4 that they need. For this task, very little > of the existing Atlas java / scala code will be changed. The classes from in > the titan project will be moved verbatim into the new project. The only > other changes will be the addition of the new classes that implement the > abstraction layer interfaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-122) Support for Deletion of Entities
[ https://issues.apache.org/jira/browse/ATLAS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-122: --- Fix Version/s: 0.7-incubating > Support for Deletion of Entities > > > Key: ATLAS-122 > URL: https://issues.apache.org/jira/browse/ATLAS-122 > Project: Atlas > Issue Type: New Feature >Reporter: Suma Shivaprasad > Assignee: David Kantor > Fix For: 0.7-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (ATLAS-916) Return system attributes in get entity definition
[ https://issues.apache.org/jira/browse/ATLAS-916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-916: --- Comment: was deleted (was: [~svimal2106] You've attached a patch but the item is still marked open. Is your fix ready for review? If so, please change the status of this item to Patch Available, and create a review board request with your changes so they can be reviewed by the community. Thanks...) > Return system attributes in get entity definition > - > > Key: ATLAS-916 > URL: https://issues.apache.org/jira/browse/ATLAS-916 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Vimal Sharma > Attachments: ATLAS-916.patch > > > Atlas should maintain system attributes created time, last modified time, > created by user and last modified by user for every entity. This information > should be returned in get entity definition -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-916) Return system attributes in get entity definition
[ https://issues.apache.org/jira/browse/ATLAS-916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15529590#comment-15529590 ] David Kantor commented on ATLAS-916: [~svimal2106] You've attached a patch but the item is still marked open. Is your fix ready for review? If so, please change the status of this item to Patch Available, and create a review board request with your changes so they can be reviewed by the community. Thanks... > Return system attributes in get entity definition > - > > Key: ATLAS-916 > URL: https://issues.apache.org/jira/browse/ATLAS-916 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Vimal Sharma > Attachments: ATLAS-916.patch > > > Atlas should maintain system attributes created time, last modified time, > created by user and last modified by user for every entity. This information > should be returned in get entity definition -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-1035) In some scenarios, creation of 2 instances with references between them in one call fails
[ https://issues.apache.org/jira/browse/ATLAS-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor resolved ATLAS-1035. - Resolution: Not A Bug Assignee: David Kantor I have debugged this issue, and Atlas is working as designed. In Atlas, the value of a non-composite reference is the referenced object's ID. The target of a non-composite reference is not implicitly created by Atlas if it does not already exist. When setting a non-composite reference, either the target object must already exist, or, the target must be created in the same Atlas entity operation as the referencing object by either: * being specified as one of the "root" entities when calling the Atlas operation, or * being the target of a composite reference from some other entity that is being created If neither of these is true, and the target of a non-composite reference does not exist, then the Atlas entity operation will fail with exception you encountered, because Atlas is saying: "This is a non-composite reference, but the target object has no existing vertex in the graph, and is not being created in this operation, so I don't know what ID to use for setting the reference". To avoid the "EntityNotFoundException: Could not find vertex for Class Reference" error, ensure that both entities are explicitly specified when calling the Atlas entities operation. The other option is changing the model, such that the reference is defined as composite. Note that this option has other ramifications, as it means the fully inflated target object will be returned whenever the source object is queried, and, the target object will be automatically deleted when the source object is deleted. Feel free to ping me if you'd like to discuss this further. > In some scenarios, creation of 2 instances with references between them in > one call fails > - > > Key: ATLAS-1035 > URL: https://issues.apache.org/jira/browse/ATLAS-1035 > Project: Atlas > Issue Type: Bug >Reporter: Prasad S Madugundu >Assignee: David Kantor > > If I create 2 instances that have references between them in one call, it > fails with the following exception: > SEVERE: org.apache.atlas.typesystem.exception.EntityNotFoundException: Could > not find vertex for Class Reference (type: > ConnectorDirectoryModel_OMRSConnectorDirectory, id: ) > Caused by: org.apache.atlas.repository.RepositoryException: > org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find > vertex for Class Reference (type: > ConnectorDirectoryModel_OMRSConnectorDirectory, id: ) > at > org.apache.atlas.repository.graph.GraphBackedMetadataRepository.createEntities(GraphBackedMetadataRepository.java:137) > at > org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:42) > at > org.apache.atlas.services.DefaultMetadataService.createEntities(DefaultMetadataService.java:337) > at > org.apache.atlas.services.DefaultMetadataService.createEntities(DefaultMetadataService.java:333) > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:125) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) > at > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) > at > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) > at > com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) > at > com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) > at > com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) > at > com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) > at > com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) > at > com.sun.jersey.ser
[jira] [Commented] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417717#comment-15417717
]
David Kantor commented on ATLAS-1117:
-
[~suma.shivaprasad] Jeff's patch has been verified (see earlier comments and
attached logs). I can commit his change or would you like to do that? Please
advise.
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jeffrey Hagelberg
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: application.log, quick_start.log, rb50983.patch
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Co
[jira] [Updated] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1117:
Attachment: application.log
Attached server application.log from test of Jeff's patch.
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jeffrey Hagelberg
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: application.log, quick_start.log, rb50983.patch
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHa
[jira] [Updated] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1117:
Attachment: quick_start.log
+1 for patch
Verified Jeff's patch with embedded HBase/Solr distribution. Atlas server and
embedded HBase and Solr started successfully, and quick_start ran without
error. Server startup console output below. See attached log for quick_start
output.
{noformat}
<1:493> % source ./conf/atlas-env.sh;bin/atlas_start.py
configured for local hbase.
hbase started.
configured for local solr.
solr started.
setting up solr collections...
Apache Atlas Server started!!!
{noformat}
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jeffrey Hagelberg
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: quick_start.log, rb50983.patch
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
[jira] [Commented] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417696#comment-15417696
]
David Kantor commented on ATLAS-1117:
-
[~suma.shivaprasad] I am testing Jeff's patch and will post results and vote.
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jeffrey Hagelberg
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: rb50983.patch
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHa
[jira] [Updated] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-1117:
Assignee: Jeffrey Hagelberg (was: David Kantor)
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Jeffrey Hagelberg
>Priority: Blocker
> Fix For: 0.8-incubating
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> at org.eclipse.jetty.server.Server.doStart(Server.java:354)
> at
>
[jira] [Commented] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417260#comment-15417260
]
David Kantor commented on ATLAS-1117:
-
[~shwethags] This may be related to [~jnhagelb]'s changes for ATLAS-693. I
will start an initial investigation as Jeff is tied up with other work at the
moment. He will pick this up ASAP if it's confirmed that 693 caused the
regression.
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
> Assignee: David Kantor
>Priority: Blocker
> Fix For: 0.8-incubating
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(Containe
[jira] [Assigned] (ATLAS-1117) Atlas start fails on trunk
[
https://issues.apache.org/jira/browse/ATLAS-1117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor reassigned ATLAS-1117:
---
Assignee: David Kantor
> Atlas start fails on trunk
> --
>
> Key: ATLAS-1117
> URL: https://issues.apache.org/jira/browse/ATLAS-1117
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
> Assignee: David Kantor
>Priority: Blocker
> Fix For: 0.8-incubating
>
>
> {noformat}
> 2016-08-11 15:19:02,407 WARN - [main:] ~ FAILED
> org.eclipse.jetty.server.Server@5d9b7a8a: java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> (AbstractLifeCycle:212)
> java.lang.NoClassDefFoundError:
> com/thinkaurelius/titan/diskstorage/locking/LocalLockMediators
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:469)
> at
> com.thinkaurelius.titan.diskstorage.hbase.HBaseStoreManager.openDatabase(HBaseStoreManager.java:457)
> at
> com.thinkaurelius.titan.diskstorage.Backend.getStandaloneGlobalConfiguration(Backend.java:387)
> at
> com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.(GraphDatabaseConfiguration.java:1277)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:93)
> at
> com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:73)
> at
> org.apache.atlas.repository.graphdb.titan0.Titan0Database.getGraphInstance(Titan0Database.java:114)
> at
> org.apache.atlas.repository.graph.TitanGraphProvider.getGraphInstance(TitanGraphProvider.java:43)
> at
> org.apache.atlas.repository.graph.GraphHelper.(GraphHelper.java:74)
> at
> org.apache.atlas.repository.graph.DeleteHandler.(DeleteHandler.java:54)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:115)
> at
> org.apache.atlas.RepositoryMetadataModule.getDeleteHandlerImpl(RepositoryMetadataModule.java:151)
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:98)
> at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> at
> com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> at com.google.inject.spi.Elements.getElements(Elements.java:110)
> at
> com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:84)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:77)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:131)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> at org.eclipse.jetty.server.Server.doStart(Server.java:354)
> at
>
[jira] [Updated] (ATLAS-991) Lower bound checking not always disabled for Entities being deleted
[
https://issues.apache.org/jira/browse/ATLAS-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-991:
---
Attachment: (was: rb49869.patch)
> Lower bound checking not always disabled for Entities being deleted
> ---
>
> Key: ATLAS-991
> URL: https://issues.apache.org/jira/browse/ATLAS-991
> Project: Atlas
> Issue Type: Bug
>Reporter: Jeffrey Hagelberg
> Assignee: David Kantor
> Attachments: rb49869.patch
>
>
> It is often the case multiple entities get deleted at the same time. The
> current delete logic does not compute the delete closure up front, so it is
> possible for the processing of a particular Entity to trigger a lower bound
> violation in an entity that will be deleted but has not been processed yet.
> This issue is specific to hard delete. It does not happen when soft delete
> is being used.
> Here's a simple test that can be added to
> GraphBackedMetadataRepositoryDeleteTestBase to reproduce the issue:
> {noformat}
> @Test
> public void testLowerBoundsIgnoredOnDeletedEntities() throws Exception {
>
> String hrDeptGuid = createHrDeptGraph();
> ITypedReferenceableInstance hrDept =
> repositoryService.getEntityDefinition(hrDeptGuid);
> Map<String, String> nameGuidMap = getEmployeeNameGuidMap(hrDept);
> ITypedReferenceableInstance john =
> repositoryService.getEntityDefinition(nameGuidMap.get("John"));
> String johnGuid = john.getId()._getId();
> ITypedReferenceableInstance max =
> repositoryService.getEntityDefinition(nameGuidMap.get("Max"));
> String maxGuid = max.getId()._getId();
>
> ITypedReferenceableInstance jane =
> repositoryService.getEntityDefinition(nameGuidMap.get("Jane"));
> String janeGuid = jane.getId()._getId();
>
> //the lower bound constraint on jane.subordinates should not be
> enforced since jane is being deleted
> deleteEntities(johnGuid, maxGuid, janeGuid);
>
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-991) Lower bound checking not always disabled for Entities being deleted
[
https://issues.apache.org/jira/browse/ATLAS-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-991:
---
Attachment: rb49869.patch
> Lower bound checking not always disabled for Entities being deleted
> ---
>
> Key: ATLAS-991
> URL: https://issues.apache.org/jira/browse/ATLAS-991
> Project: Atlas
> Issue Type: Bug
>Reporter: Jeffrey Hagelberg
> Assignee: David Kantor
> Attachments: rb49869.patch
>
>
> It is often the case multiple entities get deleted at the same time. The
> current delete logic does not compute the delete closure up front, so it is
> possible for the processing of a particular Entity to trigger a lower bound
> violation in an entity that will be deleted but has not been processed yet.
> This issue is specific to hard delete. It does not happen when soft delete
> is being used.
> Here's a simple test that can be added to
> GraphBackedMetadataRepositoryDeleteTestBase to reproduce the issue:
> {noformat}
> @Test
> public void testLowerBoundsIgnoredOnDeletedEntities() throws Exception {
>
> String hrDeptGuid = createHrDeptGraph();
> ITypedReferenceableInstance hrDept =
> repositoryService.getEntityDefinition(hrDeptGuid);
> Map<String, String> nameGuidMap = getEmployeeNameGuidMap(hrDept);
> ITypedReferenceableInstance john =
> repositoryService.getEntityDefinition(nameGuidMap.get("John"));
> String johnGuid = john.getId()._getId();
> ITypedReferenceableInstance max =
> repositoryService.getEntityDefinition(nameGuidMap.get("Max"));
> String maxGuid = max.getId()._getId();
>
> ITypedReferenceableInstance jane =
> repositoryService.getEntityDefinition(nameGuidMap.get("Jane"));
> String janeGuid = jane.getId()._getId();
>
> //the lower bound constraint on jane.subordinates should not be
> enforced since jane is being deleted
> deleteEntities(johnGuid, maxGuid, janeGuid);
>
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-1109) DSL support for Map and Array property search
[ https://issues.apache.org/jira/browse/ATLAS-1109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Kantor updated ATLAS-1109: Description: Add DSL support for searching array and map attributes. Perhaps there could be a contains predicate for searching array attributes for specific value(s), or searching a map attribute for specific key(s). Issue Type: New Feature (was: Bug) Summary: DSL support for Map and Array property search (was: Add ability to search on Map and Array properties for an entity) > DSL support for Map and Array property search > - > > Key: ATLAS-1109 > URL: https://issues.apache.org/jira/browse/ATLAS-1109 > Project: Atlas > Issue Type: New Feature >Reporter: Neeru Gupta > > Add DSL support for searching array and map attributes. Perhaps there could > be a contains predicate for searching array attributes for specific value(s), > or searching a map attribute for specific key(s). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-991) Lower bound checking not always disabled for Entities being deleted
[
https://issues.apache.org/jira/browse/ATLAS-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-991:
---
Attachment: rb49869.patch
Rebased and addressed review comments. [~shwethags] Kindly review latest
changes on review board. If there are no further comments, please commit the
patch. Thanks...
> Lower bound checking not always disabled for Entities being deleted
> ---
>
> Key: ATLAS-991
> URL: https://issues.apache.org/jira/browse/ATLAS-991
> Project: Atlas
> Issue Type: Bug
>Reporter: Jeffrey Hagelberg
> Assignee: David Kantor
> Attachments: rb49869.patch
>
>
> It is often the case multiple entities get deleted at the same time. The
> current delete logic does not compute the delete closure up front, so it is
> possible for the processing of a particular Entity to trigger a lower bound
> violation in an entity that will be deleted but has not been processed yet.
> This issue is specific to hard delete. It does not happen when soft delete
> is being used.
> Here's a simple test that can be added to
> GraphBackedMetadataRepositoryDeleteTestBase to reproduce the issue:
> {noformat}
> @Test
> public void testLowerBoundsIgnoredOnDeletedEntities() throws Exception {
>
> String hrDeptGuid = createHrDeptGraph();
> ITypedReferenceableInstance hrDept =
> repositoryService.getEntityDefinition(hrDeptGuid);
> Map<String, String> nameGuidMap = getEmployeeNameGuidMap(hrDept);
> ITypedReferenceableInstance john =
> repositoryService.getEntityDefinition(nameGuidMap.get("John"));
> String johnGuid = john.getId()._getId();
> ITypedReferenceableInstance max =
> repositoryService.getEntityDefinition(nameGuidMap.get("Max"));
> String maxGuid = max.getId()._getId();
>
> ITypedReferenceableInstance jane =
> repositoryService.getEntityDefinition(nameGuidMap.get("Jane"));
> String janeGuid = jane.getId()._getId();
>
> //the lower bound constraint on jane.subordinates should not be
> enforced since jane is being deleted
> deleteEntities(johnGuid, maxGuid, janeGuid);
>
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-991) Lower bound checking not always disabled for Entities being deleted
[
https://issues.apache.org/jira/browse/ATLAS-991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Kantor updated ATLAS-991:
---
Attachment: (was: rb49869.patch)
> Lower bound checking not always disabled for Entities being deleted
> ---
>
> Key: ATLAS-991
> URL: https://issues.apache.org/jira/browse/ATLAS-991
> Project: Atlas
> Issue Type: Bug
>Reporter: Jeffrey Hagelberg
> Assignee: David Kantor
>
> It is often the case multiple entities get deleted at the same time. The
> current delete logic does not compute the delete closure up front, so it is
> possible for the processing of a particular Entity to trigger a lower bound
> violation in an entity that will be deleted but has not been processed yet.
> This issue is specific to hard delete. It does not happen when soft delete
> is being used.
> Here's a simple test that can be added to
> GraphBackedMetadataRepositoryDeleteTestBase to reproduce the issue:
> {noformat}
> @Test
> public void testLowerBoundsIgnoredOnDeletedEntities() throws Exception {
>
> String hrDeptGuid = createHrDeptGraph();
> ITypedReferenceableInstance hrDept =
> repositoryService.getEntityDefinition(hrDeptGuid);
> Map<String, String> nameGuidMap = getEmployeeNameGuidMap(hrDept);
> ITypedReferenceableInstance john =
> repositoryService.getEntityDefinition(nameGuidMap.get("John"));
> String johnGuid = john.getId()._getId();
> ITypedReferenceableInstance max =
> repositoryService.getEntityDefinition(nameGuidMap.get("Max"));
> String maxGuid = max.getId()._getId();
>
> ITypedReferenceableInstance jane =
> repositoryService.getEntityDefinition(nameGuidMap.get("Jane"));
> String janeGuid = jane.getId()._getId();
>
> //the lower bound constraint on jane.subordinates should not be
> enforced since jane is being deleted
> deleteEntities(johnGuid, maxGuid, janeGuid);
>
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 49869: ATLAS-991: avoid enforcing required attributes on deleted entities
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49869/ --- (Updated Aug. 9, 2016, 12:36 a.m.) Review request for atlas. Bugs: ATLAS-991 https://issues.apache.org/jira/browse/ATLAS-991 Repository: atlas Description --- When deleting entities, avoid enforcing lower bound constraints (i.e. required attributes) on all entities being deleted. Previously, only the entities that were specifically requested for deletion were skipped when enforcing required attributes, but any composite entities that were owned by 'root' deletion candidates were not skipped, causing NullRequiredAttributeException to be thrown unnecessarily. Diffs (updated) - repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 8d31c1b079a11291a87569fde6f2cb21d4258ca9 repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java e301a00b16ad316a85714cbda4af43d01f11f4ad repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 0a90bda72e90569c1d9d8fdef2c9df9827742874 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java 34842c3dc2904b123fecb5087aee357678b00156 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java cc6026461bcd38b83dde0bcec2e835f5824e894f repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java 90bb635126a02cd38c045bd5332499c4481d3751 repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java 8a0a6bc60818c3dd58e12386c55a471de2508b5e Diff: https://reviews.apache.org/r/49869/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
[jira] [Commented] (ATLAS-1106) Fix Build failure due to wrong version in graphdb/common pom
[
https://issues.apache.org/jira/browse/ATLAS-1106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412675#comment-15412675
]
David Kantor commented on ATLAS-1106:
-
+1 for the patch
> Fix Build failure due to wrong version in graphdb/common pom
>
>
> Key: ATLAS-1106
> URL: https://issues.apache.org/jira/browse/ATLAS-1106
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1106.patch
>
>
> {noformat}
> mvn clean install
> [INFO] Scanning for projects...
> [ERROR] [ERROR] Some problems were encountered while processing the POMs:
> [FATAL] Non-resolvable parent POM for
> org.apache.atlas:atlas-graphdb-common:[unknown-version]: Could not find
> artifact org.apache.atlas:atlas-graphdb:pom:0.8-incubator-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 18, column 13
> [WARNING] 'build.plugins.plugin.version' for
> org.codehaus.mojo:keytool-maven-plugin is missing. @
> org.apache.atlas:atlas-webapp:[unknown-version],
> /Users/sshivaprasad/workspace-latest/incubator-atlas-ws1/webapp/pom.xml, line
> 361, column 21
> @
> [ERROR] The build could not read 1 project -> [Help 1]
> [ERROR]
> [ERROR] The project org.apache.atlas:atlas-graphdb-common:[unknown-version]
> (/Users/sshivaprasad/workspace-latest/incubator-atlas-ws1/graphdb/common/pom.xml)
> has 1 error
> [ERROR] Non-resolvable parent POM for
> org.apache.atlas:atlas-graphdb-common:[unknown-version]: Could not find
> artifact org.apache.atlas:atlas-graphdb:pom:0.8-incubator-SNAPSHOT and
> 'parent.relativePath' points at wrong local POM @ line 18, column 13 -> [Help
> 2]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
> [ERROR] [Help 2]
> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Review Request 49869: ATLAS-991: avoid enforcing required attributes on deleted entities
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49869/ --- (Updated Aug. 8, 2016, 10:39 p.m.) Review request for atlas. Changes --- Added test case per Shwetha's review comment. Bugs: ATLAS-991 https://issues.apache.org/jira/browse/ATLAS-991 Repository: atlas Description --- When deleting entities, avoid enforcing lower bound constraints (i.e. required attributes) on all entities being deleted. Previously, only the entities that were specifically requested for deletion were skipped when enforcing required attributes, but any composite entities that were owned by 'root' deletion candidates were not skipped, causing NullRequiredAttributeException to be thrown unnecessarily. Diffs (updated) - repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java 8d31c1b079a11291a87569fde6f2cb21d4258ca9 repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java e301a00b16ad316a85714cbda4af43d01f11f4ad repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 0a90bda72e90569c1d9d8fdef2c9df9827742874 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java 34842c3dc2904b123fecb5087aee357678b00156 repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java cc6026461bcd38b83dde0bcec2e835f5824e894f repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java 90bb635126a02cd38c045bd5332499c4481d3751 repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java 8a0a6bc60818c3dd58e12386c55a471de2508b5e Diff: https://reviews.apache.org/r/49869/diff/ Testing --- Ran all unit and integration tests with no regressions. Thanks, David Kantor
Re: Review Request 47466: ATLAS-693: Add titan 0.5.4 implementation of graph database abstraction layer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/47466/#review145010 --- Ship it! Ship It! - David Kantor On Aug. 5, 2016, 10:19 p.m., Jeff Hagelberg wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/47466/ > --- > > (Updated Aug. 5, 2016, 10:19 p.m.) > > > Review request for atlas and David Kantor. > > > Bugs: ATLAS-693 > https://issues.apache.org/jira/browse/ATLAS-693 > > > Repository: atlas > > > Description > --- > > Adding titan 0.5.4 implementation of abstraction layer under graphdb/titan0. > > - The original TitanGraphProvider class has been morphed into Titan0Database > in this new project. This class has the logic that configures Titan and > loads the graph. A new TitanGraphProvider has been introduced in > atlas-repository that delegates everything to Titan0Database. Now, the > TitanGraph is being initialized using the code in the new abstraction layer > implementation. > > - The atlas-titan project has been merged into graphdb/titan0. This > consolidates all of the titan 0.5.4 configuration logic into the new project. > All dependencies on atlas-titan have been replaced with dependencies on > atlas-graphdb-titan0. The Atlas war now bundles atlas-graphdb-api and > atlas-graphdb-titan0. At this point, though, Atlas is still not using the > abstraction layer classes (AtlasVertex, AtlasEdge, etc) at runtime. It is > still hard-coded to use the Titan 0.5.4 classes. The only logic in the > abstraction layer implementation that is actually being used is the logic to > configure Titan and create a TitanGraph. > > - start_atlas.py was updated to remove atlas-titan from the classpath. All > of the titan-specific classes now live in the uber "atlas-graphdb-titan0" > jar. The pom.xml for that project has special logic to guarantee that the > customized titan classes get included in the jar rather than the originals, > so it is no longer necessary to force the custom titan classes to be near the > front of the classpath. > > - Added "solr" as a supported key for the solr index. This is the key that > is used for the index in Titan 1.0.0 (which has solr support built-in). > solr5 is still allowed for backward compatibility, but this allows more > consistency between the implementations and simplifies the test setup logic > in the pom.xmls (we will need to update fewer properties when testing titan > 1.0.0). > - I also included in here the change to allow additional jvm arguments to be > passed when running the tests. This can be used, for example, to attach > jprofiler to the jvm running the tests. > - Rebased on 8/4/2016 > > > Diffs > - > > .gitignore 0086a69ba29e82dd826f83170a6cc73648c76de5 > catalog/pom.xml 5946c10da788871021c997b91e83103e2aa094dc > distro/src/bin/atlas_start.py bb6189886fe34e7e4496b521cd5cae62df492e36 > distro/src/test/python/scripts/TestMetadata.py > f943b8afa5464ce14653912ecc9e2f105402d24b > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdge.java > 71b577b5c2e8aee8fec816b9224672ebc2cad5b3 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdgeDirection.java > e7da1c9a5075004973ba68b8a36a2c95756e0e62 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasElement.java > 3c41693e531eb523f04a5c5781f1f123ab38ef07 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java > f312117af9cb6ae8de21d565bfcf3b10d9bdfcc6 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphIndex.java > PRE-CREATION > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java > 2776b2242c4469dcae344616ccfe100633a79432 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java > f0dca9a28eebd8996d55d71dcc3254cce9cd4098 > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java > 7ee6b2831a7610e7d5b1d36ae69282a9315dc06b > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasPropertyKey.java > PRE-CREATION > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasSchemaViolationException.java > fda83b807f3640c04a660001941376f41be6d9be > > graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVert
Re: Review Request 49869: ATLAS-991: avoid enforcing required attributes on deleted entities
> On Aug. 4, 2016, 7:17 a.m., Shwetha GS wrote: > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java, > > line 886 > > <https://reviews.apache.org/r/49869/diff/4/?file=1448757#file1448757line886> > > > > Add tests with lower bounds on map as well I will add a testcase as suggested. - David --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49869/#review144724 ------- On Aug. 5, 2016, 3:12 a.m., David Kantor wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49869/ > --- > > (Updated Aug. 5, 2016, 3:12 a.m.) > > > Review request for atlas. > > > Bugs: ATLAS-991 > https://issues.apache.org/jira/browse/ATLAS-991 > > > Repository: atlas > > > Description > --- > > When deleting entities, avoid enforcing lower bound constraints (i.e. > required attributes) on all entities being deleted. Previously, only the > entities that were specifically requested for deletion were skipped when > enforcing required attributes, but any composite entities that were owned by > 'root' deletion candidates were not skipped, causing > NullRequiredAttributeException to be thrown unnecessarily. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java > 8d31c1b079a11291a87569fde6f2cb21d4258ca9 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java > e301a00b16ad316a85714cbda4af43d01f11f4ad > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 81fb76e27019012fada88a70a13b4c5264c236bf > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java > 34842c3dc2904b123fecb5087aee357678b00156 > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java > cc6026461bcd38b83dde0bcec2e835f5824e894f > > repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java > 90bb635126a02cd38c045bd5332499c4481d3751 > > repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java > 8a0a6bc60818c3dd58e12386c55a471de2508b5e > > Diff: https://reviews.apache.org/r/49869/diff/ > > > Testing > --- > > Ran all unit and integration tests with no regressions. > > > Thanks, > > David Kantor > >
