Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review220053
---


Ship it!




Ship It!

- Sarath Subramanian


On March 20, 2020, 7:06 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 20, 2020, 7:06 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/import/template
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 
>   repository/pom.xml 802d587 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f07 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 9625f94 
>   repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> 759dcdf 
>   repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
>   repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 
> 
> 
> Diff: https://reviews.apache.org/r/71482/diff/20/
> 
> 
> Testing
> ---
> 
> Tested both the endpoint with curl call.
> 
> Tested upload term csv with around 100 records from curl.
> 
> 
> Thanks,
> 
> mayank jain
> 
>

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review220048
---


Ship it!




Ship It!

- Sidharth Mishra


On March 20, 2020, 2:06 p.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 20, 2020, 2:06 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/import/template
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 
>   repository/pom.xml 802d587 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f07 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 9625f94 
>   repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> 759dcdf 
>   repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
>   repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 
> 
> 
> Diff: https://reviews.apache.org/r/71482/diff/20/
> 
> 
> Testing
> ---
> 
> Tested both the endpoint with curl call.
> 
> Tested upload term csv with around 100 records from curl.
> 
> 
> Thanks,
> 
> mayank jain
> 
>

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 20, 2020, 2:06 p.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

All the mentioned changes has been addresed in this patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/import/template

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/20/

Changes: https://reviews.apache.org/r/71482/diff/19-20/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
> > Lines 570 (patched)
> > 
> >
> > Please remove these kind of extra blank lines from here as well all 
> > other places

I can still see the extra blank lines other places. Kindly let me know if there 
is any specific reason you add them?


> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
> > Lines 690 (patched)
> > 
> >
> > Instead of \n use System.lineSeparator(). Please do this at all other 
> > places in entire patch

Please mention the reason for drop.


> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/util/FileUtils.java
> > Lines 99 (patched)
> > 
> >
> > Try to create a const for 0 explaining what does the position contains

Please mention the reason for drop.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219988
---


On March 19, 2020, 5:38 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 19, 2020, 5:38 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/import/template
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 
>   repository/pom.xml 802d587 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f07 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 9625f94 
>   repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> 759dcdf 
>   repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
>   repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 
> 
> 
> Diff:

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 19, 2020, 5:38 a.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

The mentioned review comments are been addresed in the provided patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/import/template

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/19/

Changes: https://reviews.apache.org/r/71482/diff/18-19/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219988
---




repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 570 (patched)


Please remove these kind of extra blank lines from here as well all other 
places



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 649 (patched)


Do something like:

Set ret = null;

if (StringUtils.isNotBlank(csvRecord)) {
  ...
  ret = new HashSet();
  ...
}

return ret;



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 672 (patched)


there is on extra space after system msg and then you are again doing + " 
". Please remove one. Same is there at other places. please take care of this 
at all other locations



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 682 (patched)


you are already passing atlasGlossaryTerm as input to this function where 
you are using set of this object and then retunring the same object 
atlasGlossaryTerm and assigning it to same atlasGlossaryTerm at 573. Please 
revisit the design and rename the function accordingly. The function should 
always do one thing and should be named accordingly. Depending on that you 
should design the input and output arguments



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 690 (patched)


Instead of \n use System.lineSeparator(). Please do this at all other 
places in entire patch



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 50 (patched)


Please call readCSV function directly instead of using class name as its in 
the same class



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 99 (patched)


Try to create a const for 0 explaining what does the position contains



repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java
Lines 960 (patched)


Please remove the extra space:

assertEquals(e.getMessage(), "No Data found in the uploaded file!");


- Sidharth Mishra


On March 13, 2020, 1 p.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 13, 2020, 1 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> retu

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 13, 2020, 1 p.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/18/

Changes: https://reviews.apache.org/r/71482/diff/17-18/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 13, 2020, 12:47 p.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/17/

Changes: https://reviews.apache.org/r/71482/diff/16-17/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 13, 2020, 9:11 a.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

The mentioned review comments are been addressed in this patch and also this 
patch consists of only Server side changes 
while UI side changes will be handled in a seprate UI patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/16/

Changes: https://reviews.apache.org/r/71482/diff/15-16/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219912
---




repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1124 (patched)


Remove extra blank line



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 549 (patched)


Please remove the extra blank lines like this even other places.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 558 (patched)


Remove the extra blank line


- Sidharth Mishra


On March 12, 2020, 10:37 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 12, 2020, 10:37 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   dashboardv2/gruntfile.js fef4e08 
>   dashboardv2/package-lock.json 7f25b57 
>   dashboardv2/package.json e90040e 
>   dashboardv2/public/css/scss/theme.scss aad0dfa 
>   dashboardv2/public/index.html.tpl 7cd86bd 
>   dashboardv2/public/js/main.js 41af465 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e35 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d 
>   dashboardv3/package-lock.json 3918ecc 
>   dashboardv3/package.json 5dc0510 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d 
>   dashboardv3/public/index.html.tpl 1baeb12 
>   dashboardv3/public/js/main.js 96ef7ae 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c5 
>   dashboardv3/public/js/utils/UrlLinks.js 95ca065 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 12, 2020, 10:37 a.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

The mentioned review comments are been resolved in this patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  dashboardv2/gruntfile.js fef4e08 
  dashboardv2/package-lock.json 7f25b57 
  dashboardv2/package.json e90040e 
  dashboardv2/public/css/scss/theme.scss aad0dfa 
  dashboardv2/public/index.html.tpl 7cd86bd 
  dashboardv2/public/js/main.js 41af465 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/gruntfile.js f55ff0d 
  dashboardv3/package-lock.json 3918ecc 
  dashboardv3/package.json 5dc0510 
  dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
  dashboardv3/public/css/scss/theme.scss 2b0c45d 
  dashboardv3/public/index.html.tpl 1baeb12 
  dashboardv3/public/js/main.js 96ef7ae 
  dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
83da9c5 
  dashboardv3/public/js/utils/UrlLinks.js 95ca065 
  dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/15/

Changes: https://reviews.apache.org/r/71482/diff/14-15/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with a

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219893
---




repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 78 (patched)


CSVReader (line #99) reads from Reader; give this, consider avoiding 
creation of temporary file here. Instead directly read from 
uploadedInputStream, like this:

fileData = FileUtils.readCSV(uploadedInputStream);

  
  public static List readCSV(InputStream inputStream) throws 
IOException {
List ret = new ArrayList<>()
  
try (CSVReader csvReader = new CSVReader(new 
InputStreamReader(inputStream))) {
  ...
}

return ret;
  }
  
Once this refactoring is done, please remove unused methods from FileUtils.


- Madhan Neethiraj


On March 11, 2020, 12:05 p.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 11, 2020, 12:05 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   dashboardv2/gruntfile.js fef4e08 
>   dashboardv2/package-lock.json 7f25b57 
>   dashboardv2/package.json e90040e 
>   dashboardv2/public/css/scss/theme.scss aad0dfa 
>   dashboardv2/public/index.html.tpl 7cd86bd 
>   dashboardv2/public/js/main.js 41af465 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e35 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d 
>   dashboardv3/package-lock.json 3918ecc 
>   dashboardv3/package.json 5dc0510 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d 
>   dashboardv3/public/index.html.tpl 1baeb12 
>   dashboardv3/public/js/main.js 96ef7ae 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c5 
>   dashboardv3/public/js/utils/UrlLinks.js 95ca065 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
>   intg/src/main/java/org/apa

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 11, 2020, 12:05 p.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

This potch addresses the review comments.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  dashboardv2/gruntfile.js fef4e08 
  dashboardv2/package-lock.json 7f25b57 
  dashboardv2/package.json e90040e 
  dashboardv2/public/css/scss/theme.scss aad0dfa 
  dashboardv2/public/index.html.tpl 7cd86bd 
  dashboardv2/public/js/main.js 41af465 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/gruntfile.js f55ff0d 
  dashboardv3/package-lock.json 3918ecc 
  dashboardv3/package.json 5dc0510 
  dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
  dashboardv3/public/css/scss/theme.scss 2b0c45d 
  dashboardv3/public/index.html.tpl 1baeb12 
  dashboardv3/public/js/main.js 96ef7ae 
  dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
83da9c5 
  dashboardv3/public/js/utils/UrlLinks.js 95ca065 
  dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml a6da9b8 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/14/

Changes: https://reviews.apache.org/r/71482/diff/13-14/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records fro

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219852
---




intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 171 (patched)


"ATLAS-400-00-9B" => "ATLAS-400-00-098"



intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 229 (patched)


Duplicate error code - #228 & #229. Update to "ATLAS-500-00-016"



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 96 (patched)


Is it necessary to write to a file, then read it into List? Can't 
CSVReader (#119) directly read from InputStream?

  new CSVReader(new InputStreamReader(uploadedInputStream));



webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java
Lines 962 (patched)


Is it necessary to return a 'File' here - given the template is not a large 
string? And, how are the files returned here deleted?

Consider returning javax.ws.rs.core.StreamingOutput, like:

  public StreamingOutput produceTemplate() throws AtlasBaseException {
return new StreamingOutput() {
@Override
public void write(OutputStream output) throws IOException, 
WebApplicationException {

output.write(glossaryService.getGlossaryTermTemplateString());
}
}
  }


- Madhan Neethiraj


On March 9, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 9, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   dashboardv2/gruntfile.js fef4e08 
>   dashboardv2/package-lock.json 7f25b57 
>   dashboardv2/package.json e90040e 
>   dashboardv2/public/css/scss/theme.scss 0589e09 
>   dashboardv2/public/index.html.tpl a6a999e 
>   dashboardv2/public/js/main.js 75e16c3 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e35 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d 
>   dashboardv3/package-lock.json 3918ecc 
>   dashboardv3/package.json 5dc0510 
>   dashboardv3/public/css/scss/leftsideba

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 9, 2020, 10:07 a.m.)


Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

The mentioned review comments are been resolved in this patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  dashboardv2/gruntfile.js fef4e08 
  dashboardv2/package-lock.json 7f25b57 
  dashboardv2/package.json e90040e 
  dashboardv2/public/css/scss/theme.scss 0589e09 
  dashboardv2/public/index.html.tpl a6a999e 
  dashboardv2/public/js/main.js 75e16c3 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/gruntfile.js f55ff0d 
  dashboardv3/package-lock.json 3918ecc 
  dashboardv3/package.json 5dc0510 
  dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
  dashboardv3/public/css/scss/theme.scss 2b0c45d 
  dashboardv3/public/index.html.tpl 2edbb65 
  dashboardv3/public/js/main.js 26fd709 
  dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
83da9c5 
  dashboardv3/public/js/utils/UrlLinks.js 2bbe679 
  dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml f76c6a0 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/13/

Changes: https://reviews.apache.org/r/71482/diff/12-13/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with ar

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219788
---



- Sidharth Mishra


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e3540 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d5e 
>   dashboardv3/package-lock.json 3918eccaa 
>   dashboardv3/package.json 5dc05104f 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
>   dashboardv3/public/index.html.tpl 2edbb659d 
>   dashboardv3/public/js/main.js 26fd70991 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c57c 
>   dashboardv3/public/js/utils/UrlLinks.js 2bbe6796f 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e4a 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50dca 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a08e 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514bc2 
>   pom.xml f76c6a05e 
>   repository/pom.xml 802d587a8 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d58 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f073f 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 9625f9409 
>   repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> 759dcdf42 
>   repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219787
---




repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Line 74 (original), 83 (patched)


Remove extra space after entityChangeNotifier



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 89 (patched)


Align the = with other assignments in constructors



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1094 (patched)


Remove this extra line



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1099 (patched)


Remove the extra new line here



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1100 (patched)


It's better to append the IOExcpetion message to AtlasBaseException for 
more details of exception



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1101 (patched)


Template -> template



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1106 (patched)


Remove this extra line



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1108 (patched)


please make this as well local to try



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1109 (patched)


This can be made local to try block and final



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1125 (patched)


Please remove extra line. Same is present other places as well.



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1127 (patched)


An Error - Error



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1128 (patched)


An Error - Error



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1139 (patched)


remove extrra line



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1143 (patched)


An Error - Error



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1144 (patched)


An Error - Error



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 532 (patched)


We should always try to avoid function argument to be out argument. It will 
be hard understand and its error prone. Here the second argument 
failedTermMsgList is an out argument. Please avoid this.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 555 (patched)


Please Fix the indentation



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 559 (patched)


Please Remove extra blank line



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 560 (patched)


Better change this if else to ->

if(GlossaryService.isNameInvalid(glossaryName)) {
   // Error
}else{
  // Success
}



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 567 (patched)


Please fix Indentation



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 586 (patched)


Better use System.getProperty("line.separator") instead of '\n'. Please 
check the indentation here as well



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 587 (patched)


It seems failedTermMsgList is not being used outside of this function. We 
can remove this argument from the function. This will also avoid out argument



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 676 (patched)

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Sidharth Mishra]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219786
---




common/src/main/java/org/apache/atlas/repository/Constants.java
Lines 205 (patched)


Please provide comments which will be useful to get some more context or 
remove the comment. In this case the variable name is more informative than the 
comment.



common/src/main/java/org/apache/atlas/repository/Constants.java
Lines 207 (patched)


Please consider renaming GlossaryImportSupportedFileFormats to 
GlossaryImportSupportedFileExtensions as you have used extensions at other 
places and comments



dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js
Lines 53 (patched)


This comment is not adding any useful information as the initialize block 
is self explanatory.


- Sidharth Mishra


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e3540 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d5e 
>   dashboardv3/package-lock.json 3918eccaa 
>   dashboardv3/package.json 5dc05104f 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
>   dashboardv3/public/index.html.tpl 2edbb659d 
>   dashboardv3/public/js/main.js 26fd70991 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c57c 
>   dashboardv3/public/js/utils/UrlLinks.js 2bbe6796f 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/Gloss

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219772
---




intg/src/main/java/org/apache/atlas/AtlasConfiguration.java
Line 66 (original), 66 (patched)


Empty string as default value for 'atlas.import.temp.directory' is 
by-design. Why should the default be changed?



intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 171 (patched)


"File Type" => "file type"



intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 195 (patched)


"No Data" => "No data"



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1107 (patched)


Avoid prefix 'atlas' to variables/members.



repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1122 (patched)


Contents of failedTermMsgList don't seem to be used here; how are the 
failures reported back to the caller?



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 59 (patched)


- which directory does this file, named 'template', get created? In webapp 
directory or the directory from which Atlas server was launched?
- when does this file get cleaned up (i.e. deleted)?
- if this method is called simultaneously from mutiple threads, the file 
contents will be overwritten. What is the impact?



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 86 (patched)


Error message doesn't have enough details to troubleshoot the issue. Please 
review and update.



webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java
Lines 957 (patched)


I suggest to use replace '/template' with '/import/template'.


- Madhan Neethiraj


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboa

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated March 4, 2020, 10:07 a.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

UI changes addded for Beta UI and Error handling support added in UI


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
  dashboardv2/gruntfile.js fef4e08c3 
  dashboardv2/package-lock.json 7f25b5752 
  dashboardv2/package.json e90040edb 
  dashboardv2/public/css/scss/theme.scss 0589e0920 
  dashboardv2/public/index.html.tpl a6a999e53 
  dashboardv2/public/js/main.js 75e16c3aa 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
1fa1e3540 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/gruntfile.js f55ff0d5e 
  dashboardv3/package-lock.json 3918eccaa 
  dashboardv3/package.json 5dc05104f 
  dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
  dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
  dashboardv3/public/index.html.tpl 2edbb659d 
  dashboardv3/public/js/main.js 26fd70991 
  dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
83da9c57c 
  dashboardv3/public/js/utils/UrlLinks.js 2bbe6796f 
  dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e4a 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50dca 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a08e 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514bc2 
  pom.xml f76c6a05e 
  repository/pom.xml 802d587a8 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d58 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f073f 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
9625f9409 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
759dcdf42 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6bce 


Diff: https://reviews.apache.org/r/71482/diff/12/

Changes: https://reviews.apache.org/r/71482/diff/11-12/


Testing
---

Tested both the e

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Feb. 28, 2020, 11 a.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Rebased patch with master.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
  dashboardv2/gruntfile.js fef4e08 
  dashboardv2/package-lock.json 7f25b57 
  dashboardv2/package.json e90040e 
  dashboardv2/public/css/scss/theme.scss 0589e09 
  dashboardv2/public/index.html.tpl a6a999e 
  dashboardv2/public/js/main.js 75e16c3 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50d 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a0 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml f76c6a0 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/11/

Changes: https://reviews.apache.org/r/71482/diff/10-11/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Jan. 9, 2020, 12:37 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

This patch contains changes for creating Glossary which is not already present 
in the system.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 0b28243 
  dashboardv2/gruntfile.js 174d67f 
  dashboardv2/package-lock.json 168ea11 
  dashboardv2/package.json dd6f9a5 
  dashboardv2/public/css/scss/theme.scss 0589e09 
  dashboardv2/public/index.html.tpl a6a999e 
  dashboardv2/public/js/main.js 6ba1e12 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 4e98c41 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js c8cda4f 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 6e726f2 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 3ebd70d 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml eb2689f 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/10/

Changes: https://reviews.apache.org/r/71482/diff/9-10/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Dec. 9, 2019, 12:42 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 0b28243 
  dashboardv2/gruntfile.js 174d67f 
  dashboardv2/package-lock.json 168ea11 
  dashboardv2/package.json dd6f9a5 
  dashboardv2/public/css/scss/theme.scss 0589e09 
  dashboardv2/public/index.html.tpl a6a999e 
  dashboardv2/public/js/main.js 6ba1e12 
  dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 1fa1e35 
  dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
PRE-CREATION 
  dashboardv2/public/js/utils/UrlLinks.js 4e98c41 
  dashboardv2/public/js/views/glossary/GlossaryLayoutView.js c8cda4f 
  dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js PRE-CREATION 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 979bd0a 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 7a2aae2 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml eb2689f 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/9/

Changes: https://reviews.apache.org/r/71482/diff/8-9/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Oct. 17, 2019, 1:03 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

All the raised issues are been addresed in this patch.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/importHeaderRow

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/import


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 357affd 
  intg/src/main/java/org/apache/atlas/AtlasConfiguration.java f9d5fe4 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 9812356 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  pom.xml ce7d6fb 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/8/

Changes: https://reviews.apache.org/r/71482/diff/7-8/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review218202
---




common/src/main/java/org/apache/atlas/repository/Constants.java
Lines 206 (patched)


SupportFileFormat => GlosasaryImportSupportedFileFormats



intg/src/main/java/org/apache/atlas/ApplicationProperties.java
Lines 63 (patched)


- TEMP_FILE_LOCATION - if this is specific to glossary import, consider 
renaming to GLOSSARY_IMPORT_TEMP_DIRECTORY
- also, consider moving this to AtlasConfiguration, similar to 
IMPORT_TEMP_DIRECTORY



intg/src/main/java/org/apache/atlas/AtlasErrorCode.java
Lines 162 (patched)


ATLAS-400-00-881 => ATLAS-400-00-9B



repository/pom.xml
Lines 230 (patched)


define the version in parent pom.xml, and reference here with the property 
name - like:
  ${opencsv.version}

Please review #236, #241 as well for similar update.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 542 (patched)


Consider rewritting:
  if(!(record.length>0) || !StringUtils.isNotBlank(record[0])){
  
as:
  if (record.length < 1 || StringUtils.isBlank(record[0])) {



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 547 (patched)


'glossaryName' is likely to be referenced in multiple rows in the imported 
file. I suggest to cache the retrieved glossaryGuid (like #552) and avoid the 
call at #549.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 549 (patched)


Replace "invalidNameChars[1]" with ".", or a constant like:
 static final String SEP_TYPENAME_ATTR = ".";

review for other such use of invalidNameChars.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 572 (patched)


getGlossaryTermHeaders() => getGlossaryImportHeaderRow()



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 581 (patched)


termName => attrName



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 585 (patched)


Is it necessary to include relationship names in 'header row'? If yes, can 
you please give few example relationship names that are needed in import.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 595 (patched)


Consider using StringUtils.join(), instead of stripping first and last char 
from toString().



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 598 (patched)


populateGlossaryTermObject() seems to expect the columns in the following 
order. This order may not be the same in 'glossaryTermHeaderListAsString', 
which is created by iterating through atlasEntityDef.getAttributeDefs() and 
atlasEntityDef.getRelationshipAttributeDefs(). Please review and update; it 
will be helpful to return 'headerRow' hardcoded with the same order as expected 
in populateGlossaryTermObject().
 - glossaryName
 - termName
 - shortDescription
 - longDescription
 - examples
 - abbreviation
 - usage
 - additionalAttributes
 - tranlationTerms
 - validValuesFor
 - synonyms
 - replacedBy
 - validValues
 - replacementTerms
 - isA
 - anchor
 - classifies
 - preferredToTerms
 - perferredTerms



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 59 (patched)


- which directory would this file (named "template") be created in?
- every call to getGlossaryTermTemplate() will write the given data to the 
same file. Does it overwrite or append the contents?
- When does the "template" file get deleted?



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 69 (patched)


Consider using FilenameUtils.getExtension(fileName) - 
http://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/FilenameUtils.html#getExtension(java.lang.String)



webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java
Lines 957 (patched)


"/template" => "/importHeaderRow"



webapp/src/main/java/o

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Oct. 14, 2019, 1:21 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details also the Unit Test cases for 
both the end points.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/template

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/importGlossaryData";


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 357affd 
  intg/src/main/java/org/apache/atlas/ApplicationProperties.java d3afd53 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 9812356 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
be1698f 
  repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
  repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
  repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/7/

Changes: https://reviews.apache.org/r/71482/diff/6-7/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Oct. 7, 2019, 2:01 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

This patch consists additional implementations of excel files(xls/xlsx) support 
with same end points as of csv.


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details.

* The 1st endpoint {glossary/template} return template file this would be type 
of format of data that shows how the data needs to be populated by user in the 
file.

http://localhost:21000/api/atlas/v2/glossary/template

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/importGlossaryData";


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 357affd 
  intg/src/main/java/org/apache/atlas/ApplicationProperties.java d3afd53 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 9812356 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/6/

Changes: https://reviews.apache.org/r/71482/diff/5-6/


Testing
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

[mayank jain]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/
---

(Updated Oct. 1, 2019, 1:08 p.m.)


Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Patch for ATLAS-3346 - Import GlossaryTerms CSV into a Glossary


Summary (updated)
-

ATLAS-3423:-Import Glossary Terms CSV into a Glossary


Bugs: ATLAS-3423
https://issues.apache.org/jira/browse/ATLAS-3423


Repository: atlas


Description (updated)
---

This patch consists implementation for 2 end points first for template download 
and other for csv file upload with term details.

* The 1st endpoint {glossary/template?fileType=csv} return template file this 
would be type of format of data that shows how the data needs to be populated 
by user in the file.

http://localhost:21000/api/atlas/v2/glossary/template?fileType=csv

Template structure:-

GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, TranslatedTerms, 
IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"


* The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
inside Glossary.

curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
"Cache-Control: no-cache" -F file=@template_6.csv 
"http://localhost:21000/api/atlas/v2/glossary/importGlossaryData";


Note:-

While populating the data in the  csv file each record should be maintained in 
single Line (enter command within the record would result in parsing the second 
line as a new record).

The downloaded template needs to be saved as whateverTheFileNameIs.csv 
explicitly.

If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
returned or else List of Errors would returned for user to rectify them further.


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 2f08efc 
  intg/src/main/java/org/apache/atlas/ApplicationProperties.java d3afd53 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 9a1aa65 
  
intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
 660514b 
  repository/pom.xml 802d587 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
9229d2d 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
cdc3f07 
  repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 9625f94 
  repository/src/main/java/org/apache/atlas/util/CSVFileUtils.java PRE-CREATION 
  webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 


Diff: https://reviews.apache.org/r/71482/diff/5/

Changes: https://reviews.apache.org/r/71482/diff/4-5/


Testing (updated)
---

Tested both the endpoint with curl call.

Tested upload term csv with around 100 records from curl.


Thanks,

mayank jain