David Knupp has posted comments on this change. Change subject: IMPALA-2013: Issue Hbase queries individually during data-load. ......................................................................
Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/3728/2/bin/load-data.py File bin/load-data.py: PS2, Line 148: if 'ERROR' in stdout: > Were you able to test this path? I tested this a couple of ways -- first by running this script with HBase already stopped, and then by running it with HBase running and killing HBase midstream. Here's an example of good output: Executing HBase Command: hbase shell create 'functional_hbase.alltypes', 'd' [STDOUT]: HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 1.2.0-cdh5.9.0-SNAPSHOT, rUnknown, Tue Jun 14 13:55:25 PDT 2016 create 'functional_hbase.alltypes', 'd'0 row(s) in 2.5540 seconds Hbase::Table - functional_hbase.alltypes And here's an example of bad output: Executing HBase Command: hbase shell create 'functional_hbase.alltypesnopart', 'd' [STDOUT]: HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 1.2.0-cdh5.9.0-SNAPSHOT, rUnknown, Tue Jun 14 13:55:25 PDT 2016 create 'functional_hbase.alltypesnopart', 'd' ERROR: KeeperErrorCode = ConnectionLoss for /hbase Here is some help for this command: Creates a table. Pass a table name, and a set of column family specifications (at least one), and, optionally, table configuration. Column specification can be a simple string (name), or a dictionary (dictionaries are described below in main help output), necessarily including NAME attribute. Examples: Create a table with namespace=ns1 and table qualifier=t1 hbase> create 'ns1:t1', {NAME => 'f1', VERSIONS => 5} [...] It seems to be consistent, thought the "help" text might differ. STDERR tends to be hundreds and hundreds of not very useful java exception output. PS2, Line 149: error = [l.strip() for l in stdout.split('\n') if 'ERROR' in l][0] > I think this is sufficiently complex for you to provide in comments what st Done PS2, Line 154: except IOError: : LOGGER.error("Unable to excute HBase queries: couldn't open input file.") > Is this referring to a failure in the with open()? Is this at the correct i Nope -- but already fixed in patch 3. PS2, Line 304: # pdb.set_trace() > Remove Done -- To view, visit http://gerrit.cloudera.org:8080/3728 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I911d972ba8ad3a2a084c8195074556153722c7e2 Gerrit-PatchSet: 2 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: David Knupp <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Harrison Sheinblatt <[email protected]> Gerrit-Reviewer: Ishaan Joshi <[email protected]> Gerrit-Reviewer: Michael Brown <[email protected]> Gerrit-HasComments: Yes
