-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39980/
-----------------------------------------------------------
Review request for Ambari, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
Bugs: AMBARI-13747
https://issues.apache.org/jira/browse/AMBARI-13747
Repository: ambari
Description
-------
Hive View should have a widget to:
Create/Choose a database and create a table inside it
Create table using a delimited (such as CSV) file from HDFS or local file system
Discover column names if it has a header
Discover suggested data types
Create a table with a given name
Optionally create an internal table that is ORC by default
Implementation Steps :
User uploads a CSV file.
Server reads the input stream and without saving parses it to generate preview
rows and detect datatypes of columns
Server sends table info to the UI.
User can change column names and datatypes and submits table info.
Server creates a table using hive job, and returns job info back.
UI keeps looping for job success.
On Job success UI sends the same file again.
If required the file’s columns are extracted and rest of the input stream is
given to the HDFS Api for uploading to the hive metastore.
Server returns success or exception.
Changes :
ColumnDescription : added enum for hive data types.?
ConnectionFactory : Added HdfsApi for connection to HDFS.
package.json : added dependency ember-cli-uploader
bower.json : added dependency ember-uploader": "0.3.9"
view.xml : added resource upload for UploadService
Additions :
resources/uploads: new packages for upload related classes
ColumnDescriptionImpl : another impl ColumnDescription for better
handling of data
IParser : Interface for parsers
DataParser : Decorator Parser for hiding internal impls.
CSVParser : for parsing CSV using apache commons-csv
QueryGenerator : class for generating HQL query from given inputs.
UploadService : REST interface for front end
DataParserTest : the test case class for DataParser.
UI
router.js : added route /upload-table
/app/adapters :
file-upload : ember uploader customized
upload-table : adapter for REST calls
/controllers :
upload-table : controller for handling all activities in
Upload Table tab
/templates :
upload-table : template for UI on Upload Tables tab
Minor changes :
Row : added equals and hashcode toString
ParseOptions : Parsing options to be passed to parser
ParseUtils : Utility class
TableInfo : Input to QueryGenerator’s method
UI
file-upload
navbar-widget : now includes one more tab for upload table
i18n.js : added constants
app.scss : minor change
constants.js : added constants
Diffs
-----
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ColumnDescription.java
d7ea560
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/ConnectionFactory.java
82ac1f5
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/client/Row.java
306530a
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/CSVParser.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/ColumnDescriptionImpl.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/DataParser.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/IParser.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/ParseOptions.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/ParseUtils.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/QueryGenerator.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/TableInfo.java
PRE-CREATION
contrib/views/hive/src/main/java/org/apache/ambari/view/hive/resources/uploads/UploadService.java
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/adapters/file-upload.js
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/adapters/upload-table.js
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/components/file-upload.js
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/components/navbar-widget.js
c3659cf
contrib/views/hive/src/main/resources/ui/hive-web/app/controllers/upload-table.js
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/initializers/i18n.js
5ae9b7e
contrib/views/hive/src/main/resources/ui/hive-web/app/router.js 5a51b11
contrib/views/hive/src/main/resources/ui/hive-web/app/styles/app.scss bf271c4
contrib/views/hive/src/main/resources/ui/hive-web/app/templates/upload-table.hbs
PRE-CREATION
contrib/views/hive/src/main/resources/ui/hive-web/app/utils/constants.js
a349d51
contrib/views/hive/src/main/resources/ui/hive-web/bower.json d43881f
contrib/views/hive/src/main/resources/ui/hive-web/package.json 6ecdcb6
contrib/views/hive/src/main/resources/view.xml fdc32d7
contrib/views/hive/src/test/java/org/apache/ambari/view/hive/resources/upload/DataParserTest.java
PRE-CREATION
Diff: https://reviews.apache.org/r/39980/diff/
Testing
-------
DataParserTest added for testing the functionality of DataParser class.
Manual testing of overall feature, including file uploads in HDFS.
Thanks,
Nitiraj Rathore
