Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19754/ --- (Updated April 8, 2014, 6:27 p.m.) Review request for hive. Changes --- addressing review comments. - move to hcatalog - expose HiveConf to client API Bugs: HIVE-5687 https://issues.apache.org/jira/browse/HIVE-5687 Repository: hive-git Description --- Defines an API for streaming data into Hive using ACID support. Diffs (updated) - hcatalog/pom.xml 50ce296 hcatalog/streaming/pom.xml PRE-CREATION hcatalog/streaming/src/docs/package.html PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ConnectionError.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HeartBeatFailure.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ImpersonationFailed.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidColumn.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidPartition.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTable.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTrasactionState.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/PartitionCreationFailed.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/QueryFailedException.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/SerializationError.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingException.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingIOFailure.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatchUnAvailable.java PRE-CREATION hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionError.java PRE-CREATION hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/StreamingIntegrationTester.java PRE-CREATION hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestDelimitedInputWriter.java PRE-CREATION hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java PRE-CREATION hcatalog/streaming/src/test/sit PRE-CREATION metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1bbe02e packaging/pom.xml de9b002 packaging/src/main/assembly/src.xml bdaa47b Diff: https://reviews.apache.org/r/19754/diff/ Testing --- Unit tests included. Also done manual testing by streaming data using flume. Thanks, Roshan Naik
Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19754/#review39817 --- hcatalog/streaming/pom.xml https://reviews.apache.org/r/19754/#comment72461 typo: artifectId should be artifactId hcatalog/streaming/pom.xml https://reviews.apache.org/r/19754/#comment72462 typo: artifectId should be artifactId hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java https://reviews.apache.org/r/19754/#comment72463 suggestion for Txnid: either spell out transaction (transaction ID -- preferable) or use capital I like the parameter (TxnId) hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java https://reviews.apache.org/r/19754/#comment72464 Why does the parameter name have both-caps ID for maxTxnID while it's init-cap Id for minTxnId? Are parameter names case-sensitive? Also a suggestion for Txnid in description: either spell out transaction (transaction ID -- preferable) or use capital ID like the parameter (TxnID). hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java https://reviews.apache.org/r/19754/#comment72465 Same question as line 108 about minTxnId vs maxTxnID capitalization hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72520 Nit: period at the end (next line too) hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72466 Editorial nits: Please capitalize nulls and end the second sentence with a period (next line) just for consistency with the first sentence. hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72467 Grammar nit: Remove s from indicates because the subjects are plural. hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72468 Consistency nit: Since other param descriptions are capitalized on the first word, please do the same here. Bonus points if you capitalize all the param descriptions in this patch, but I'm not going to comment on all of them. You could argue for a rule that only capitalizes full sentences and proper nouns like Hive, in which case [pun alert] it's okay to leave input uncapitalized. But I favor visual consistency over rule consistency, except when I'm inconsistent. Terminal periods aren't essential (given the typical style of javadocs) but they're recommended when a description has multiple sentences. Hm, but that's inconsistent with my visual consistency preference. Why am I wasting your time with this trivia? hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72517 should endpoint be explained? (your call) hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72469 Editorial nit: non existing seems okay in this context, but nonexistent is the real word (your choice). Consistency nit again: Since other exception descriptions are capitalized on the first word, please do the same here. hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72470 ditto line 57 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72471 ditto line 58 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72472 ditto line 59 (capitalization) hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72474 ditto line 60 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72473 Hive nit: please capitalize hive Editorial nits: please capitalize a and perhaps spell out configuration in conf object unless conf is the proper term for the object hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72475 ditto line 65 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java https://reviews.apache.org/r/19754/#comment72516 ditto line 59 hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19754/ --- (Updated April 1, 2014, 11:53 p.m.) Review request for hive. Changes --- updating patch Bugs: HIVE-5687 https://issues.apache.org/jira/browse/HIVE-5687 Repository: hive-git Description --- Defines an API for streaming data into Hive using ACID support. Diffs (updated) - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1bbe02e packaging/pom.xml de9b002 packaging/src/main/assembly/src.xml bdaa47b pom.xml 7343683 streaming/pom.xml PRE-CREATION streaming/src/java/org/apache/hive/streaming/AbstractRecordWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/ConnectionError.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/DelimitedInputWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/HeartBeatFailure.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/HiveEndPoint.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/ImpersonationFailed.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidColumn.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidPartition.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidTable.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidTrasactionState.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/PartitionCreationFailed.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/QueryFailedException.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/RecordWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/SerializationError.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingConnection.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingException.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingIOFailure.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionBatch.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionBatchUnAvailable.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionError.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/StreamingIntegrationTester.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/TestDelimitedInputWriter.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/TestStreaming.java PRE-CREATION streaming/src/test/sit PRE-CREATION Diff: https://reviews.apache.org/r/19754/diff/ Testing --- Unit tests included. Also done manual testing by streaming data using flume. Thanks, Roshan Naik
Review Request 19754: Defines a api for streaming data into Hive using ACID support.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/19754/ --- Review request for hive. Bugs: HIVE-5687 https://issues.apache.org/jira/browse/HIVE-5687 Repository: hive-git Description --- Defines an API for streaming data into Hive using ACID support. Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 209349b packaging/pom.xml de9b002 packaging/src/main/assembly/src.xml bdaa47b pom.xml 7343683 streaming/pom.xml PRE-CREATION streaming/src/java/org/apache/hive/streaming/AbstractLazySimpleRecordWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/ConnectionError.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/DelimitedInputWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/HeartBeatFailure.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/HiveEndPoint.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/ImpersonationFailed.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidColumn.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidPartition.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidTable.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/InvalidTrasactionState.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/PartitionCreationFailed.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/QueryFailedException.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/RecordWriter.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/SerializationError.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingConnection.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingException.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/StreamingIOFailure.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionBatch.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionBatchUnAvailable.java PRE-CREATION streaming/src/java/org/apache/hive/streaming/TransactionError.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/StreamingIntegrationTester.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/TestDelimitedInputWriter.java PRE-CREATION streaming/src/test/org/apache/hive/streaming/TestStreaming.java PRE-CREATION streaming/src/test/sit PRE-CREATION Diff: https://reviews.apache.org/r/19754/diff/ Testing --- Unit tests included. Also done manual testing by streaming data using flume. Thanks, Roshan Naik
