[GitHub] [atlas] waterWang opened a new pull request #117: add oracle repository

2020-12-25 Thread GitBox


waterWang opened a new pull request #117:
URL: https://github.com/apache/atlas/pull/117


   “add oracle repository” to make ‘mvn clean -DskipTests install’ success



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] waterWang opened a new pull request #116: Hotfix#1

2020-12-25 Thread GitBox


waterWang opened a new pull request #116:
URL: https://github.com/apache/atlas/pull/116


   update  ‘http://repo.typesafe.com/typesafe/releases/’  to  'https', the 
‘http://repo.typesafe.com/typesafe/releases/’  Connection refused



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nikhilbonte commented on pull request #115: ATLAS-3913 Swagger documentation for APIs

2020-12-02 Thread GitBox


nikhilbonte commented on pull request #115:
URL: https://github.com/apache/atlas/pull/115#issuecomment-737264113


   Closing as created new review board item with updated patch - 
https://reviews.apache.org/r/73057



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nikhilbonte closed pull request #115: ATLAS-3913 Swagger documentation for APIs

2020-12-02 Thread GitBox


nikhilbonte closed pull request #115:
URL: https://github.com/apache/atlas/pull/115


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nikhilbonte opened a new pull request #115: ATLAS-3913 Swagger documentation for APIs

2020-11-23 Thread GitBox


nikhilbonte opened a new pull request #115:
URL: https://github.com/apache/atlas/pull/115


   Currently, swagger documentation is not available with packaging.
   This change will allow atlas server deployment to host its REST API doc 
using swagger.
   
   Approach:
   
   1. Update enuntiate plugin version from 2.11.1 to 2.13.2
   2. Download latest swagger-ui from 
[v3.36.2.zip](https://github.com/swagger-api/swagger-ui/archive/v3.36.2.zip) & 
extract.
   3. Move swagger-ui-3.36.2/dist/* to build-tools/src/main/resources/ui-dist
   4. Update swagger tag in enuntiate.xml
   5. Inside build-tools/src/main/resources/ui-dist :
* Create index.js
* Modify SwaggerUIBundle object in ui-dist/index.html to load index.js
> url to load swagger.json
> Add requestInterceptor
   6. Updated Header.js, UrlLinks.js, Header.html  to add API documentation 
menu.
   
   How to navigate to the documentation page?
   -> Once Atlas UI is loaded, goto  **Logged-in-User -> Help -> API 
Documentation**
   https://user-images.githubusercontent.com/44048490/99964008-a62afe80-2db8-11eb-8ecd-cea617fe7f4a.png;>
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] alexwang789 opened a new pull request #114: Update LoginProcessor.java

2020-11-15 Thread GitBox


alexwang789 opened a new pull request #114:
URL: https://github.com/apache/atlas/pull/114


   Fix the bug that Atlas can't find the vaule of kerberos config ,because of 
path problem.
   According to the user's manual  from http://atlas.apache.org/, properties  
for  kerberos  should be:
   atlas.authentication.method.kerberos.principal=
   atlas.authentication.method.kerberos.keytab =
   
   But , in the codes, paths are below:
   public static final String AUTHENTICATION_PRINCIPAL = 
ATLAS_AUTHENTICATION_PREFIX+ "principal";
   public static final String AUTHENTICATION_KEYTAB = 
ATLAS_AUTHENTICATION_PREFIX+ "keytab";
   
   that are :atlas.authentication.principal&  atlas.authentication.keytab
   
   so, the properties for kerberos  couldn't be get correctly!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] mneethiraj commented on pull request #54: ATLAS-3219: New REST APIs for serviceType.

2020-10-26 Thread GitBox


mneethiraj commented on pull request #54:
URL: https://github.com/apache/atlas/pull/54#issuecomment-71660


   > Guys, what is serviceType? There is not a single piece of docs about it
   
   serviceType is a grouping of type-defs in Atlas. For example: all type-defs 
for Hive (hive_db, hive_table, hive_column, hive_process, ..) will have 
serviceType as 'hive'. This enables Atlas UI to filter type-defs list for 
specific serviceTypes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] kell18 commented on pull request #54: ATLAS-3219: New REST APIs for serviceType.

2020-10-22 Thread GitBox


kell18 commented on pull request #54:
URL: https://github.com/apache/atlas/pull/54#issuecomment-714371664


   Guys, what is serviceType? There is not a single piece of docs about it



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] gm19900510 opened a new pull request #113: Release 2.1.0 rc3 chd 6.3.1 hylink

2020-10-18 Thread GitBox


gm19900510 opened a new pull request #113:
URL: https://github.com/apache/atlas/pull/113


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] HeChuanXUPT opened a new pull request #112: update doc for janusgraph urls

2020-09-17 Thread GitBox


HeChuanXUPT opened a new pull request #112:
URL: https://github.com/apache/atlas/pull/112


   update doc for janusgraph urls
   
   404 - Not found
   https://docs.janusgraph.org/latest/elasticsearch.html
   https://docs.janusgraph.org/latest/hbase.html#_remote_server_mode_2
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] Bing-ok opened a new pull request #111: ATLAS-3906: fix dependency conflict issue

2020-09-16 Thread GitBox


Bing-ok opened a new pull request #111:
URL: https://github.com/apache/atlas/pull/111


   [ATLAS-3906](https://issues.apache.org/jira/browse/ATLAS-3906) fix 
dependency conflict issue for org.apache.curator:curator-client:jar



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] zyyBjtu commented on pull request #108: ATLAS-3923: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-09-15 Thread GitBox


zyyBjtu commented on pull request #108:
URL: https://github.com/apache/atlas/pull/108#issuecomment-693133698


   Apache JARA: https://issues.apache.org/jira/browse/ATLAS-3923



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] zyyBjtu commented on pull request #107: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-09-15 Thread GitBox


zyyBjtu commented on pull request #107:
URL: https://github.com/apache/atlas/pull/107#issuecomment-693130625


   Sorry, I just saw the message。You can use this link to access Apache JIRA.
   https://issues.apache.org/jira/browse/ATLAS-3923
   
   
   
   
   --原始邮件--
   发件人: "Sarath Subramanian"

[GitHub] [atlas] nixonrodrigues merged pull request #110: ATLAS-3940 : Upgrade snakeyaml to a version without CVE-2017-18640

2020-09-15 Thread GitBox


nixonrodrigues merged pull request #110:
URL: https://github.com/apache/atlas/pull/110


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nixonrodrigues commented on pull request #110: ATLAS-3940 : Upgrade snakeyaml to a version without CVE-2017-18640

2020-09-14 Thread GitBox


nixonrodrigues commented on pull request #110:
URL: https://github.com/apache/atlas/pull/110#issuecomment-691860982


   CI passed.
   
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/15/console
   
   +1 for PR, @crazylab , Thanks for PR.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] crazylab opened a new pull request #110: Upgrade snakeyaml to a version without CVE-2017-18640

2020-09-07 Thread GitBox


crazylab opened a new pull request #110:
URL: https://github.com/apache/atlas/pull/110


   Maven package `cassandra-all` has transitive dependency on 
`org.yaml:snakeyaml:1.11` which has 
CVE-2017-18640:https://nvd.nist.gov/vuln/detail/CVE-2017-18640
   Raised a PR to Cassandra to upgrade the version of `snakeyaml` to `1.26` 
where the CVE got fixed : https://github.com/apache/cassandra/pull/736
   Upgrade to the latest version once `cassandra-all` new release become 
available



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] crazylab closed pull request #109: Upgrade snakeyaml to a version without CVE-2017-18640

2020-09-07 Thread GitBox


crazylab closed pull request #109:
URL: https://github.com/apache/atlas/pull/109


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] crazylab opened a new pull request #109: Upgrade snakeyaml to a version without CVE-2017-18640

2020-09-07 Thread GitBox


crazylab opened a new pull request #109:
URL: https://github.com/apache/atlas/pull/109


   - Maven package cassandra-all has transitive dependency on 
org.yaml:snakeyaml:1.11 which has 
CVE-2017-18640:https://nvd.nist.gov/vuln/detail/CVE-2017-18640
   - Raised a PR to Cassandra to upgrade the version of snakeyaml to 1.26 where 
the CVE got fixed : https://github.com/apache/cassandra/pull/736
   - Upgrade to the latest version of cassandra-all once new releases become 
available



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on pull request #107: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-08-24 Thread GitBox


sarathsubramanian commented on pull request #107:
URL: https://github.com/apache/atlas/pull/107#issuecomment-679227067


   @zyyBjtu , thanks for PR. Can you please create a Apache JIRA with the 
details and attach link to PR.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] zyyBjtu opened a new pull request #108: ATLAS-3923: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-08-19 Thread GitBox


zyyBjtu opened a new pull request #108:
URL: https://github.com/apache/atlas/pull/108


   fix NullPointException in HBaseBasedAuditRepository.listEventsV2



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] zyyBjtu closed pull request #107: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-08-19 Thread GitBox


zyyBjtu closed pull request #107:
URL: https://github.com/apache/atlas/pull/107


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] zyyBjtu opened a new pull request #107: fix NullPointException in HBaseBasedAuditRepository.listEventsV2

2020-08-19 Thread GitBox


zyyBjtu opened a new pull request #107:
URL: https://github.com/apache/atlas/pull/107


   In debug mode, the audit page reports a NullPointException



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] Aka-shi commented on pull request #101: add-dist-cassandra-elasticsearch-profile and support GS

2020-08-04 Thread GitBox


Aka-shi commented on pull request #101:
URL: https://github.com/apache/atlas/pull/101#issuecomment-668440631


   @mohamed-a-abdelaziz thanks. This worked for me. I checked out onto your 
branch and was able to build this. Any idea when this PR will be merged to 
master?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] Aka-shi removed a comment on pull request #101: add-dist-cassandra-elasticsearch-profile and support GS

2020-08-03 Thread GitBox


Aka-shi removed a comment on pull request #101:
URL: https://github.com/apache/atlas/pull/101#issuecomment-668012977


   @mohamed-a-abdelaziz I've tried those earlier and again clean installed 
everything(cassandra, es) to follow your steps. But still I get `503-problem 
accessing /.` because of service unavailable. I don't see anything in logs too 
apart from Apache atlas server started. Can you help me here please?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] Aka-shi commented on pull request #101: add-dist-cassandra-elasticsearch-profile and support GS

2020-08-03 Thread GitBox


Aka-shi commented on pull request #101:
URL: https://github.com/apache/atlas/pull/101#issuecomment-668012977


   @mohamed-a-abdelaziz I've tried those earlier and again clean installed 
everything(cassandra, es) to follow your steps. But still I get `503-problem 
accessing /.` because of service unavailable. I don't see anything in logs too 
apart from Apache atlas server started. Can you help me here please?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] medb opened a new pull request #106: ATLAS-3908: Update Kafka to 2.3.1 version

2020-08-01 Thread GitBox


medb opened a new pull request #106:
URL: https://github.com/apache/atlas/pull/106


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] mohamed-a-abdelaziz commented on pull request #101: add-dist-cassandra-elasticsearch-profile and support GS

2020-07-31 Thread GitBox


mohamed-a-abdelaziz commented on pull request #101:
URL: https://github.com/apache/atlas/pull/101#issuecomment-667252655


   > @mohamed-a-abdelaziz any idea what are the steps required to run atlas 
with local cassandra and local elastic search? Don't seem to find anything in 
docs.
   
   just follow these steps 
   
   #download cassandra
   `wget 
https://mirrors.gethosted.online/apache/cassandra/3.11.7/apache-cassandra-3.11.7-bin.tar.gz`
   `tar -xzf apache-cassandra-3.11.7-bin.tar.gz`
   `#set start_rpc: true  on apache-cassandra/conf/cassandra.yml file `
   #run cassandra locally
   `/bin/cassandra -f &`
   #download elasticsearch
   `wget 
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.8.1-linux-x86_64.tar.gz`
   `tar -xzf elasticsearch-7.8.1-linux-x86_64.tar.gz`
   #run elassticsearch 
   `/bin/elasticsearch &`
   #run kafka 
   `wget 
https://www.apache.org/dyn/closer.cgi?path=/kafka/2.5.0/kafka_2.12-2.5.0.tgz`
   `tar -xzf kafka_2.12-2.5.0.tgz`
   `/bin/zookeeper-server-start.sh config/zookeeper.properties &`
   `/bin/kafka-server-start.sh config/server.properties &`
   #clone atlas from my fork if you didn't yet :)
   `#set atlas.kafka.zookeeper.connect=localhost:2181 and  
atlas.kafka.bootstrap.servers=localhost:9092 on 
distro/src/conf/atlas-application.properties (or your own ports)`
   #build atlas
   `cd `
   `mvn clean -DskipTests package -Pdist,cassandra-elasticsearch`
   `cd distro/target/apache-atlas-2.1.0-server/apache-atlas-2.1.0`
   #run atlas
   `./bin/atlas_start.py`
   #open your browser and type localhost:21000/
   #user/pass : admin/admin
   
   
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] Aka-shi commented on pull request #101: add-dist-cassandra-elasticsearch-profile and support GS

2020-07-31 Thread GitBox


Aka-shi commented on pull request #101:
URL: https://github.com/apache/atlas/pull/101#issuecomment-667081617


   @mohamed-a-abdelaziz any idea what are the steps required to run atlas with 
local cassandra and local elastic search? Don't seem to find anything in docs. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] insertmike opened a new pull request #105: Fixed small typo on type system page

2020-07-23 Thread GitBox


insertmike opened a new pull request #105:
URL: https://github.com/apache/atlas/pull/105


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] dggs123 closed pull request #104: Qualified name in terms and category headers

2020-07-17 Thread GitBox


dggs123 closed pull request #104:
URL: https://github.com/apache/atlas/pull/104


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] dggs123 opened a new pull request #104: Qualified name in terms and category headers

2020-07-17 Thread GitBox


dggs123 opened a new pull request #104:
URL: https://github.com/apache/atlas/pull/104


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] dggs123 closed pull request #103: Entities search powered directly through es

2020-07-08 Thread GitBox


dggs123 closed pull request #103:
URL: https://github.com/apache/atlas/pull/103


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] dggs123 opened a new pull request #103: Entities search powered directly through es

2020-07-08 Thread GitBox


dggs123 opened a new pull request #103:
URL: https://github.com/apache/atlas/pull/103


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] v-odier-heetch commented on pull request #102: Tdh 145

2020-06-23 Thread GitBox


v-odier-heetch commented on pull request #102:
URL: https://github.com/apache/atlas/pull/102#issuecomment-648230968


   this is a fail



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] v-odier-heetch closed pull request #102: Tdh 145

2020-06-23 Thread GitBox


v-odier-heetch closed pull request #102:
URL: https://github.com/apache/atlas/pull/102


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] v-odier-heetch commented on pull request #102: Tdh 145

2020-06-23 Thread GitBox


v-odier-heetch commented on pull request #102:
URL: https://github.com/apache/atlas/pull/102#issuecomment-648229619


   env var DRY="true" pour avoir les logs direct sur stdout sans tail puis 
bin/startAtlas



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] v-odier-heetch opened a new pull request #102: Tdh 145

2020-06-23 Thread GitBox


v-odier-heetch opened a new pull request #102:
URL: https://github.com/apache/atlas/pull/102


   hack mode



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] mohamed-a-abdelaziz opened a new pull request #101: add-dist-cassandra-elasticsearch-profile

2020-06-12 Thread GitBox


mohamed-a-abdelaziz opened a new pull request #101:
URL: https://github.com/apache/atlas/pull/101


   add-dist-cassandra-elasticsearch-profile
   update elasticsearch version



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] lyyprean commented on pull request #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-06-11 Thread GitBox


lyyprean commented on pull request #93:
URL: https://github.com/apache/atlas/pull/93#issuecomment-642469289


   
   What version does spark-atlas-connector use?
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] lyyprean commented on pull request #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-06-11 Thread GitBox


lyyprean commented on pull request #93:
URL: https://github.com/apache/atlas/pull/93#issuecomment-642468592


   what version is spark-atlas-conection used



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sekingme opened a new pull request #100: add code to config 'jetty.http.timeout' and increase the default time…

2020-05-21 Thread GitBox


sekingme opened a new pull request #100:
URL: https://github.com/apache/atlas/pull/100


   the default 'jetty.http.timeout' value 3ms can easily lead to timeout 
exception with hundreds of request. So, add code to set the idle timeout config 
and increase the default timeout seconds.
   
   
-
   at 
org.apache.atlas.web.filters.AtlasKnoxSSOAuthenticationFilter.doFilter(AtlasKnoxSSOAuthenticationFilter.java:142)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
   at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
   at 
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
   at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
   at 
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
   at 
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
   at 
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)
   at 
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)
   at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
   at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
   at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
   at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
   at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   at org.eclipse.jetty.server.Server.handle(Server.java:534)
   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
   Caused by: java.util.concurrent.TimeoutException: Idle timeout expired: 
3/3 ms
   at 
org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
   at 

[GitHub] [atlas] AmeerAssi commented on pull request #98: [ATLAS-3800] AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-629390103


   Thank you.
   Closing the PR



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi closed pull request #98: [ATLAS-3800] AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi closed pull request #98:
URL: https://github.com/apache/atlas/pull/98


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on pull request #98: [ATLAS-3800] AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


sarathsubramanian commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-629378481


   merged the changes. Please resolve this PR.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on pull request #98: [ATLAS-3800] AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-629128363


   Thank you.
   Opened this issue:
   https://issues.apache.org/jira/browse/ATLAS-3800
   
   @sarathkumarsubramanian let me know if anything else is needed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi removed a comment on pull request #98: [ATLAS-3800] AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi removed a comment on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-629108017


   Thank you.
   Opened this issue:
   https://issues.apache.org/jira/browse/ATLAS-3800
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-629108017


   Thank you.
   Opened this issue:
   https://issues.apache.org/jira/browse/ATLAS-3800
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi edited a comment on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-15 Thread GitBox


AmeerAssi edited a comment on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-627370163


   @sarathkumarsubramanian kind reminder for this review.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-14 Thread GitBox


sarathsubramanian commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-628995492


   @AmeerAssi can you create an apache jira and attach your patch in the JIRA. 
I will merge these changes. I need a apache jira id for commit message.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-14 Thread GitBox


sarathsubramanian commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-628993270


   +1



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sekingme opened a new pull request #99: optimize and normalize the hive-bridge code style

2020-05-14 Thread GitBox


sekingme opened a new pull request #99:
URL: https://github.com/apache/atlas/pull/99


   1. fix the code style problem.
   2. optimize realization of some method.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi edited a comment on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-10 Thread GitBox


AmeerAssi edited a comment on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-626381318


   > @AmeerAssi for existing deployments to add this attribute, can you please 
include a patch file to add this attribute? You can check a similar patch for 
reference - 
https://github.com/apache/atlas/blob/master/addons/models/3000-Cloud/patches/004-s3_add_attributes.json
   
   @sarathkumarsubramanian 
   AFAIK this model was never released with any Atlas version, can you please 
explain why this patch file is needed, as my understanding this patch file came 
for backward compatibility.
   
   in case this still needed, can you please explain what these properties mean 
in the mentioned json:
   "applyToVersion": "1.1",
"updateToVersion": "1.2"
   
   in my case what values should they have?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on a change in pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-10 Thread GitBox


AmeerAssi commented on a change in pull request #98:
URL: https://github.com/apache/atlas/pull/98#discussion_r422692691



##
File path: addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json
##
@@ -251,6 +251,15 @@
   "isIndexable": false,
   "isOptional":  true,
   "isUnique":false
+},
+{
+  "name":"accountId",
+  "typeName":"string",
+  "cardinality": "SINGLE",
+  "isIndexable": true,
+  "isOptional":  true,
+  "isUnique":false,
+  "searchWeight" : 5

Review comment:
   @sarathkumarsubramanian thanks for the suggestion.
   done.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-10 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-626381318


   > @AmeerAssi for existing deployments to add this attribute, can you please 
include a patch file to add this attribute? You can check a similar patch for 
reference - 
https://github.com/apache/atlas/blob/master/addons/models/3000-Cloud/patches/004-s3_add_attributes.json
   
   @sarathkumarsubramanian 
   AFAIK this model was never released with any Atlas version, can you please 
explain why this patch file is needed, as my understanding this patch file came 
for backward compatibility.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on a change in pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-10 Thread GitBox


sarathsubramanian commented on a change in pull request #98:
URL: https://github.com/apache/atlas/pull/98#discussion_r422679888



##
File path: addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json
##
@@ -251,6 +251,15 @@
   "isIndexable": false,
   "isOptional":  true,
   "isUnique":false
+},
+{
+  "name":"accountId",
+  "typeName":"string",
+  "cardinality": "SINGLE",
+  "isIndexable": true,
+  "isOptional":  true,
+  "isUnique":false,
+  "searchWeight" : 5

Review comment:
   if you want this attribute to be searchable in quicksearch/suggestions. 
Add a higher value for search weight. 9 or 10





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sarathsubramanian commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-10 Thread GitBox


sarathsubramanian commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-626367082


   @AmeerAssi for existing deployments to add this attribute, can you please 
include a patch file to add this attribute? You can check a similar patch for 
reference - 
https://github.com/apache/atlas/blob/master/addons/models/3000-Cloud/patches/004-s3_add_attributes.json



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-08 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-626037246


   Thanks @sidharthkmishra for approving.
   @sarathsubramanian @ashutoshm can you please take a look per Sidharth's.
   I am not getting any response from you in mail.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi edited a comment on pull request #98: AWS scheme is missing the aws account id that contains the S3 bucket.

2020-05-04 Thread GitBox


AmeerAssi edited a comment on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-620852203


   We are using Atlas models in our product.
   the product gives scan capabilities for customer's S3 in AWS.
   when scan result shown based on the model, it is missing for which account 
this bucket belongs. so it will be easy to access the account and review the 
scanned information by the customer.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] nixonrodrigues commented on pull request #84: ATLAS-3774 : Fix typo in authentication docs

2020-05-01 Thread GitBox


nixonrodrigues commented on pull request #84:
URL: https://github.com/apache/atlas/pull/84#issuecomment-622397725


   +1 for the fix. thanks @tartina 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] tartina commented on pull request #84: Fix typo in authentication docs

2020-05-01 Thread GitBox


tartina commented on pull request #84:
URL: https://github.com/apache/atlas/pull/84#issuecomment-622384741


   Added https://issues.apache.org/jira/browse/ATLAS-3774



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] HorizonNet commented on pull request #84: Fix typo in authentication docs

2020-04-29 Thread GitBox


HorizonNet commented on pull request #84:
URL: https://github.com/apache/atlas/pull/84#issuecomment-621228056


   Also noticed this one. @tartina You'll need a Jira first related to this PR.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi commented on pull request #98: Today AWS scheme is missing the aws account id that contains the S3 bucket.

2020-04-28 Thread GitBox


AmeerAssi commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-620852203


   So we are using Atlas models in our product.
   the product gives scan capabilities for customer's S3 in AWS.
   when scan result shown based on the model, it is missing for which account 
this bucket belongs. so it will be easy to access the account and review the 
scanned information by the customer.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] sidharthkmishra commented on pull request #98: Today AWS scheme is missing the aws account id that contains the S3 bucket.

2020-04-28 Thread GitBox


sidharthkmishra commented on pull request #98:
URL: https://github.com/apache/atlas/pull/98#issuecomment-620785613


   As per my understanding accountId is used to construct ARN and it may be 
used at AWS to distinguish resources. Could you please explain how will this be 
useful as Atlas metadata and what  is the use-case. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi opened a new pull request #98: Today AWS scheme is missing the aws account id that contains the S3 bucket.

2020-04-27 Thread GitBox


AmeerAssi opened a new pull request #98:
URL: https://github.com/apache/atlas/pull/98


   adding optional field for account id, so it will not break existing products 
that are using the current scheme, and will add ability for products to have 
the context of which account contains the bucket.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] AmeerAssi opened a new pull request #97: Today AWS scheme is missing the aws account id that contains the S3 bucket.

2020-04-27 Thread GitBox


AmeerAssi opened a new pull request #97:
URL: https://github.com/apache/atlas/pull/97


   adding optional field for account id, so it will not break existing products 
that are using the current scheme, and will add ability for products to have 
the context of which account contains the bucket.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] hsaputra opened a new pull request #96: [ATLAS-3756] Error when compile from source code - missing repo for restlet

2020-04-26 Thread GitBox


hsaputra opened a new pull request #96:
URL: https://github.com/apache/atlas/pull/96


   Error when compile from source code - can not find restlet for transitive dep
   
   Need to add repositories where restlet available.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [atlas] dwarszawski closed pull request #95: Feature/enable freetext sort

2020-04-17 Thread GitBox
dwarszawski closed pull request #95: Feature/enable freetext sort
URL: https://github.com/apache/atlas/pull/95
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] dwarszawski opened a new pull request #95: Feature/enable freetext sort

2020-04-17 Thread GitBox
dwarszawski opened a new pull request #95: Feature/enable freetext sort
URL: https://github.com/apache/atlas/pull/95
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] zhangxiong0301 opened a new pull request #75: ATLAS-3519: pool performance in hive lineage query

2020-04-09 Thread GitBox
zhangxiong0301 opened a new pull request #75: ATLAS-3519: pool performance in 
hive lineage query
URL: https://github.com/apache/atlas/pull/75
 
 
   when query lineage info for hive, the throughput reachs only 3 query per 
second.
   
   we use jstack and jfr+flame graph tools and found that the  
GremlinGroovyScriptEngine object loading class  logic takes much of the time. 
so we make the GremlinGroovyScriptEngine a global singleton object, which would 
be shared accross all query threads. finally the query performance improved 
much and got 200 qps.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] zhangxiong0301 closed pull request #75: ATLAS-3519: pool performance in hive lineage query

2020-04-09 Thread GitBox
zhangxiong0301 closed pull request #75: ATLAS-3519: pool performance in hive 
lineage query
URL: https://github.com/apache/atlas/pull/75
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] HeartSaVioR closed pull request #64: [ATLAS-3314] Loosen the relationship between spark_table and spark_storagedesc

2020-03-26 Thread GitBox
HeartSaVioR closed pull request #64: [ATLAS-3314] Loosen the relationship 
between spark_table and spark_storagedesc
URL: https://github.com/apache/atlas/pull/64
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] HeartSaVioR commented on issue #64: [ATLAS-3314] Loosen the relationship between spark_table and spark_storagedesc

2020-03-26 Thread GitBox
HeartSaVioR commented on issue #64: [ATLAS-3314] Loosen the relationship 
between spark_table and spark_storagedesc
URL: https://github.com/apache/atlas/pull/64#issuecomment-604815594
 
 
   Resolved via 
https://github.com/apache/atlas/commit/2c375b08a52ac8d1039abb4b612a41cb9d89b420


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues merged pull request #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-20 Thread GitBox
nixonrodrigues merged pull request #94: ATLAS-3665 Add 'queryText' attribute to 
the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-20 Thread GitBox
nixonrodrigues commented on issue #94: ATLAS-3665 Add 'queryText' attribute to 
the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-601792794
 
 
   Thanks @vladhlinsky for PR, 
   Thanks @HeartSaVioR @sarathsubramanian for review.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-19 Thread GitBox
sarathsubramanian commented on issue #94: ATLAS-3665 Add 'queryText' attribute 
to the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-601418486
 
 
   Thanks for the patch @vladhlinsky. Changes looks good. +1


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-19 Thread GitBox
vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 
'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-601113240
 
 
   @sarathsubramanian  thank you for the review! The PR is updated to use patch 
file.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian edited a comment on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-18 Thread GitBox
sarathsubramanian edited a comment on issue #94: ATLAS-3665 Add 'queryText' 
attribute to the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-600795336
 
 
   @vladhlinsky , can we use the patch files to add the new attribute to 
'spark_process'? Existing deployments with spark_model needs to be updated to 
add this new attribute. You can take a look at - 
https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/patches/006-kafka_topic_add_attribute.json
 for reference.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-18 Thread GitBox
sarathsubramanian commented on issue #94: ATLAS-3665 Add 'queryText' attribute 
to the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-600795336
 
 
   @vladhlinsky , can we use the patch files to add the new attribute to 
'spark_process'? Existing deployments with spark_model needs to be updated to 
add this new attribute. You can take a look at 
[](https://github.com/apache/atlas/blob/master/addons/models/1000-Hadoop/patches/006-kafka_topic_add_attribute.json)
 for reference.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-17 Thread GitBox
vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 
'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-600177724
 
 
   I checked behavior of the Spark Atlas Connector with proposed changes after 
adopting #91. Below is the list of cases when SAC creates a `spark_process` 
along with sample values of the `queryText` attribute:
   1. ML process, streaming query process
   
   **`queryText` attribute is not set.**
   
   2. `InsertIntoHiveTable`, `InsertIntoHadoopFsRelationCommand`
   
   **Tested with:**
   ```
   spark.sql("create table table_3(id int,name string)");
   spark.sql("insert into table_3 values(1,'John')");
   ```
   **`queryText`:**
   ![Screenshot from 2020-03-17 
18-32-36](https://user-images.githubusercontent.com/61428392/76878686-dc090200-687d-11ea-849f-5faba9b95855.png)
   
   3. `CreateHiveTableAsSelectCommand`, `CreateDataSourceTableAsSelectCommand`
   
   **Tested with:**
   ```
   spark.sql("create table source_table_1(col1 int,col2 string)");
   spark.sql("create table dest_table_1 as select * from source_table_1");
   ```
   **`queryText`:**
   ![Screenshot from 2020-03-17 
18-36-29](https://user-images.githubusercontent.com/61428392/76879108-71a49180-687e-11ea-9f89-707fcf6c9365.png)
   
   4. `LoadDataCommand`
   
   **Tested with:**
   ```
   import java.io.{FileOutputStream, PrintWriter}
   import java.nio.file.Files
   
   val file = Files.createTempFile("input", ".txt").toFile
   val out = new PrintWriter(new FileOutputStream(file))
   out.write("a\nb\nc\nd\n")
   out.close()
   
   spark.sql(s"CREATE TABLE hive_table_7 (name string) USING HIVE")
   spark.sql(s"LOAD DATA LOCAL INPATH '${file.getAbsolutePath}' OVERWRITE INTO 
TABLE hive_table_7")
   ```
   **`queryText`:**
   ![Screenshot from 2020-03-17 
18-38-55](https://user-images.githubusercontent.com/61428392/76879202-9862c800-687e-11ea-847c-90fda42f72fa.png)
   
   5. `InsertIntoHiveDirCommand`
   
   **Tested with:**
   ```
   spark.sql("create table overwrite_source_table_1(id int,name string)");
   spark.sql(s"insert overwrite directory 'tempDir1' select * from 
overwrite_source_table_1")
   ```
   **`queryText`:**
   ![Screenshot from 2020-03-17 
18-39-51](https://user-images.githubusercontent.com/61428392/76879274-b7f9f080-687e-11ea-938d-2f6786c09e54.png)
   
   6. `CreateViewCommand`
   
   **Tested with:**
   ```
   spark.sql("create table view_source_table(col1 int,col2 string)");
   spark.sql("create view dest_view as select * from view_source_table");
   ```
   **`queryText`:**
   ![Screenshot from 2020-03-17 
18-40-42](https://user-images.githubusercontent.com/61428392/76879391-d95adc80-687e-11ea-8709-f1071edd23e4.png)
   
   7. `SaveIntoDataSourceCommand`, `WriteToDataSourceV2Exec`
   
   **Tested with:**
   ```
   val lines = Seq("SaveIntoDataSourceCommand").toDF("line")
   val url = "jdbc:derby:memory:;databaseName=/tmp/test;create=true"
   val requiredOpts = Map("url" -> url, "dbtable" -> "lines")
   import org.apache.spark.sql.SaveMode.Overwrite
   lines.write.options(requiredOpts).format("jdbc").mode(Overwrite).save
   ```
   **`queryText` attribute is not set.**
   
   `CreateTableCommand`, `CreateDataSourceTableCommand` do not result in a 
`spark_process` being created.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues merged pull request #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-17 Thread GitBox
nixonrodrigues merged pull request #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-17 Thread GitBox
nixonrodrigues commented on issue #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#issuecomment-600130446
 
 
   Thanks @vladhlinsky for PR.
   Thanks @HeartSaVioR , @sarathsubramanian for review.
   
   Merging patch to master.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-16 Thread GitBox
sarathsubramanian commented on issue #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#issuecomment-599881646
 
 
   Thanks for the change @vladhlinsky . LGTM +1


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 'spark_process' type

2020-03-16 Thread GitBox
vladhlinsky commented on issue #94: ATLAS-3665 Add 'queryText' attribute to the 
'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-599699206
 
 
   @HeartSaVioR thank you for the review!
   After adopting #91 Spark Atlas Connector will create a `spark_process` for 
each supported at this time `QueryExecution` event except of 
`CreateTableCommand` and `CreateDataSourceTableCommand`. Most of these commands 
could be mapped to single query. Since `CreateTableCommand`, 
`CreateDataSourceTableCommand` and `ExternalCatalogEvent`'s do not result in a 
`spark_process` being created, it's possible to keep `recentQueries` attribute 
as list to aggregate corresponding SQL queries until a spark process is not 
created:
   ![Screenshot from 2020-03-16 
19-51-09](https://user-images.githubusercontent.com/61428392/76786227-8837e380-67bf-11ea-9e4d-268429a52e5d.png)
   
   But it seems that at this moment queries that are mapped to `QueryExecution` 
events will be more suitable to make `spark_process` more readable by the user.
   
   Previously, I referred to `recentQueries` attribute of the `hive_process` 
type, but as it turned out there is no case when it's value is a list of 
multiple queries:
   
https://github.com/apache/atlas/blob/master/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java#L377
   
https://github.com/apache/atlas/blob/master/addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java#L663
   
   The PR is updated to add single `queryText` attribute to the `spark_process` 
type.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-16 Thread GitBox
vladhlinsky commented on issue #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#issuecomment-599603296
 
 
   @sarathsubramanian I guess in this case the proper way will be changing 
`"relationshipCategory"` value to `"ASSOCIATION"` and removing `isContainer` 
attributes. PR is updated with these changes. Retested manually using the steps 
above.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-16 Thread GitBox
vladhlinsky commented on issue #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#issuecomment-599593065
 
 
   @sarathsubramanian sorry, i just realized that i tested proposed by you 
changes with previously existing relationship definitions and thus I missed 
that changing  "isContainer" => false leads to:
   ```
   org.apache.atlas.exception.AtlasBaseException: AGGREGATION relationshipDef 
spark_ml_model_dataset creation attempted without an end specifying isContainer
at 
org.apache.atlas.type.AtlasRelationshipType.validateAtlasRelationshipDef(AtlasRelationshipType.java:309)
at 
org.apache.atlas.type.AtlasRelationshipType.resolveReferences(AtlasRelationshipType.java:110)
at 
org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.resolveReferences(AtlasTypeRegistry.java:426)
at 
org.apache.atlas.type.AtlasTypeRegistry$AtlasTransientTypeRegistry.addTypes(AtlasTypeRegistry.java:500)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.tryTypeCreation(AtlasTypeDefGraphStore.java:1136)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.createUpdateTypesDef(AtlasTypeDefGraphStore.java:410)
at 
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
   ```
   
   changes reverted back to "isContainer" => true. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on a change in pull request #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-16 Thread GitBox
vladhlinsky commented on a change in pull request #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#discussion_r393001959
 
 

 ##
 File path: addons/models/1000-Hadoop/1100-spark_model.json
 ##
 @@ -470,6 +470,44 @@
 "cardinality": "SINGLE"
   },
   "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_model_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_model",
+"name": "dataset",
+"isContainer": true,
+"cardinality": "SINGLE"
+  },
+  "endDef2": {
+"type": "DataSet",
+"name": "model",
+"isContainer": false,
+"cardinality": "SINGLE"
+  },
+  "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_pipeline_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_pipeline",
+"name": "dataset",
+"isContainer": true,
 
 Review comment:
   Changed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on a change in pull request #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-16 Thread GitBox
vladhlinsky commented on a change in pull request #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#discussion_r393001869
 
 

 ##
 File path: addons/models/1000-Hadoop/1100-spark_model.json
 ##
 @@ -470,6 +470,44 @@
 "cardinality": "SINGLE"
   },
   "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_model_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_model",
+"name": "dataset",
+"isContainer": true,
 
 Review comment:
   Thanks! Changed.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues merged pull request #91: ATLAS-3655: Create 'spark_application' type to avoid 'spark_process' from being updated for multiple operations

2020-03-16 Thread GitBox
nixonrodrigues merged pull request #91: ATLAS-3655: Create 'spark_application' 
type to avoid 'spark_process' from being updated for multiple operations
URL: https://github.com/apache/atlas/pull/91
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues commented on issue #91: ATLAS-3655: Create 'spark_application' type to avoid 'spark_process' from being updated for multiple operations

2020-03-16 Thread GitBox
nixonrodrigues commented on issue #91: ATLAS-3655: Create 'spark_application' 
type to avoid 'spark_process' from being updated for multiple operations
URL: https://github.com/apache/atlas/pull/91#issuecomment-599382377
 
 
   @vladhlinsky ,
   
this PR has conflicts, can you please rebase with master and update PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues merged pull request #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-03-16 Thread GitBox
nixonrodrigues merged pull request #93: ATLAS-3661 Create 
'spark_column_lineage' type and relationship definition
URL: https://github.com/apache/atlas/pull/93
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] nixonrodrigues commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-03-16 Thread GitBox
nixonrodrigues commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' 
type and relationship definition
URL: https://github.com/apache/atlas/pull/93#issuecomment-599377859
 
 
   +1 for PR, thanks @vladhlinsky for PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] HeartSaVioR commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to the 'spark_process' type

2020-03-15 Thread GitBox
HeartSaVioR commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to 
the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-599359170
 
 
   How it will be changed after adopting #91? Would it be possible to couple 
query with each spark_process after #91, or we won't know about relationship 
between spark_process and the query? 
   
   If it's former, ideally `spark_process` would have single query instead of 
queries. If it's latter, recentQueries should be added into `spark_application` 
instead of `spark_process`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to the 'spark_process' type

2020-03-13 Thread GitBox
vladhlinsky commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to 
the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-598756880
 
 
   cc @HeartSaVioR @sarathsubramanian


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky opened a new pull request #94: ATLAS-3665 Add 'recentQueries' attribute to the 'spark_process' type

2020-03-13 Thread GitBox
vladhlinsky opened a new pull request #94: ATLAS-3665 Add 'recentQueries' 
attribute to the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94
 
 
   ## What changes were proposed in this pull request?
   
   Add `recentQueries` attribute to the `spark_process` type in order to make 
`spark_process` more readable by the user. The `recentQueries` attribute stores 
exact SQL quries that are executed within Spark session.
   
   ## How was this patch tested?
   
   Manually using modified version of Spark Atlas Connector:
   - Install and start Atlas.
   - Stop Atlas.
   - `1100-spark_model.json` is updated with proposed changes.
   - Start Atlas.
   - Executed the next statements using spark-shell:
   ```
   spark.sql("create table source_2_77_10(id int, new_id int, name string)");
   spark.sql("create table ctas_2_77_10 as select id as new_id from 
source_2_77_10");
   ```
   - Verified that corresponding 'spark_process' is created and has 
`recentQueries`:
   ```
   create table source_2_77_10(id int, new_id int, name string
   create table ctas_2_77_10 as select id as new_id from source_2_77_10
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to the 'spark_process' type

2020-03-13 Thread GitBox
vladhlinsky commented on issue #94: ATLAS-3665 Add 'recentQueries' attribute to 
the 'spark_process' type
URL: https://github.com/apache/atlas/pull/94#issuecomment-598756809
 
 
   Attaching screenshots.
   ![Screenshot from 2020-03-13 
15-28-00](https://user-images.githubusercontent.com/61428392/76631507-7ac5f380-654a-11ea-92f5-c03a12d678b7.png)
   ![Screenshot from 2020-03-13 
15-28-13](https://user-images.githubusercontent.com/61428392/76631501-7699d600-654a-11ea-8aee-bb8d47d08a54.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian commented on a change in pull request #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-12 Thread GitBox
sarathsubramanian commented on a change in pull request #89: ATLAS-3646 Create 
new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#discussion_r391790871
 
 

 ##
 File path: addons/models/1000-Hadoop/1100-spark_model.json
 ##
 @@ -470,6 +470,44 @@
 "cardinality": "SINGLE"
   },
   "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_model_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_model",
+"name": "dataset",
+"isContainer": true,
+"cardinality": "SINGLE"
+  },
+  "endDef2": {
+"type": "DataSet",
+"name": "model",
+"isContainer": false,
+"cardinality": "SINGLE"
+  },
+  "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_pipeline_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_pipeline",
+"name": "dataset",
+"isContainer": true,
 
 Review comment:
   same here = "isContainer" => false


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] sarathsubramanian commented on a change in pull request #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-12 Thread GitBox
sarathsubramanian commented on a change in pull request #89: ATLAS-3646 Create 
new 'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#discussion_r391790569
 
 

 ##
 File path: addons/models/1000-Hadoop/1100-spark_model.json
 ##
 @@ -470,6 +470,44 @@
 "cardinality": "SINGLE"
   },
   "propagateTags": "NONE"
+},
+{
+  "name": "spark_ml_model_dataset",
+  "serviceType": "spark",
+  "typeVersion": "1.0",
+  "relationshipCategory": "AGGREGATION",
+  "endDef1": {
+"type": "spark_ml_model",
+"name": "dataset",
+"isContainer": true,
 
 Review comment:
   "isContainer" should be false here
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] HeartSaVioR commented on issue #89: ATLAS-3646 Create new 'spark_ml_model_dataset', 'spark_ml_pipeline_dataset' relationship defs

2020-03-11 Thread GitBox
HeartSaVioR commented on issue #89: ATLAS-3646 Create new 
'spark_ml_model_dataset','spark_ml_pipeline_dataset' relationship defs
URL: https://github.com/apache/atlas/pull/89#issuecomment-598001082
 
 
   @sarathsubramanian Kindly reminder.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-03-11 Thread GitBox
vladhlinsky commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' 
type and relationship definition
URL: https://github.com/apache/atlas/pull/93#issuecomment-597892210
 
 
   cc @HeartSaVioR @sarathsubramanian


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [atlas] vladhlinsky commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' type and relationship definition

2020-03-11 Thread GitBox
vladhlinsky commented on issue #93: ATLAS-3661 Create 'spark_column_lineage' 
type and relationship definition
URL: https://github.com/apache/atlas/pull/93#issuecomment-597892124
 
 
   Attaching screenshots.
   ![Screenshot from 2020-03-11 
23-39-45](https://user-images.githubusercontent.com/61428392/76467148-a386a580-63f1-11ea-9df0-9507e015c7a6.png)
   ![Screenshot from 2020-03-10 
21-51-16](https://user-images.githubusercontent.com/61428392/76467046-66baae80-63f1-11ea-877c-cd64b874421d.png)
   ![Screenshot from 2020-03-10 
21-51-26](https://user-images.githubusercontent.com/61428392/76467051-69b59f00-63f1-11ea-8d22-f147c066a9e6.png)
   ![Screenshot from 2020-03-10 
21-51-47](https://user-images.githubusercontent.com/61428392/76467056-6de1bc80-63f1-11ea-8afd-7ce905b93797.png)
   ![Screenshot from 2020-03-11 
23-40-34](https://user-images.githubusercontent.com/61428392/76467202-bf8a4700-63f1-11ea-88c9-8320901a41b3.png)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


  1   2   3   4   >