[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385980#comment-16385980
]
George Smith commented on HIVE-14342:
-
[~ngangam] Thank you for your response!
Are you able to reproduce my problem on your cluster configuration as well?
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383719#comment-16383719
]
Naveen Gangam commented on HIVE-14342:
--
[~cabot] Your testcase is slightly different than the issue in this jira. You
are running beeline as a background process that has to use the
{{-Djline.terminal=jline.UnsupportedTerminal}} property. Otherwise there is
contention for STDOUT/STDERR streams between the JLine Terminal and Unix
background process. As a result, the backgrounded beeline process is SUSPENDED
by the OS (similar to CNTRL Z).
So without using this property, beeline wouldnt even run as a background
process. But unfortunately, it appears to have this side effect. I can look at
the jline code again to see if there have been any fixes.
Does that make sense?
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378643#comment-16378643
]
George Smith commented on HIVE-14342:
-
Anyone can confirm my issue?
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374205#comment-16374205
]
George Smith commented on HIVE-14342:
-
Are you sure this has been fixed? We are using Cloudera distribution with
HIVE-14342 bundled and I can confirm that beeline file contains check for
*Test case:*
_*test.hql content:*_
{code}
SELECT 1;
{code}
{code}
$ beeline -u jdbc:hive2://xxx:1 --silent=true --showWarnings=false
--showNestedErrs=false --showHeader=false --outputformat=tsv2 -f ~/test.hql &
wait
[1] 3076126
null1
{code}
I also tried to add this line before calling beeline:
{code}
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS
-Djline.terminal=jline.UnsupportedTerminal"
{code}
Nothing has changed.
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.1.1, 2.2.0
>
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417553#comment-15417553
]
Mohit Sabharwal commented on HIVE-14342:
Thanks, [~ngangam], LGTM +1
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410524#comment-15410524
]
Hive QA commented on HIVE-14342:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822344/HIVE-14342.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10440 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/793/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/793/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-793/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822344 - PreCommit-HIVE-MASTER-Build
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410183#comment-15410183
]
Naveen Gangam commented on HIVE-14342:
--
Thanks Mohit. I ran the following test on bash 4.1.2 (tried both conditions,
the second condition for Bash 4.2+ does not seem to work as expected).
The prompt variable does not seem to be set in any scenario. I ran the test
both in background and foreground and the result is the same. Running from a
remote node results in the same.
{code}
# cat test.sh
#!/usr/bin/env bash
if [ -z $PS1 ] # no prompt?
#if [ -v PS1 ] # On Bash 4.2+ ...
then
echo "non-interactive3"
else
echo "interactive3"
fi
# ./test.sh
non-interactive3
# ./test.sh &
[1] 5715
# non-interactive3
[1]+ Done./test.sh
{code}
>From a remote node
{code}
ngangam-MBP-2:~ ngangam$ ssh -l root "/root/test.sh"
root@'s password:
non-interactive3
{code}
In the link above, further down on the page, there is a sample using exactly
what the patch is, "-p /dev/stdin". Do you know any other means that might
work? Thanks
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-14342.2.patch, HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409377#comment-15409377
]
Naveen Gangam commented on HIVE-14342:
--
[~mohitsabharwal] [~aihuaxu] Could you please review the fix?
The proposed fix adds an additional condition to only use the
UnsupportedTerminal for beeline running as background process when run locally.
The condition detects that beeline is being invoked from a remote shell by
checking if stdin for the script is a pipe.
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408918#comment-15408918
]
Hive QA commented on HIVE-14342:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822205/HIVE-14342.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10440 tests
executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
TestQueryLifeTimeHook - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_orc_llap_counters
org.apache.hadoop.hive.metastore.txn.TestCompactionTxnHandler.testRevokeTimedOutWorkers
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/781/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/781/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-781/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12822205 - PreCommit-HIVE-MASTER-Build
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Attachments: HIVE-14342.patch, HIVE-14342.patch
>
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408628#comment-15408628
]
Hive QA commented on HIVE-14342:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12822163/HIVE-14342.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/777/testReport
Console output:
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/777/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-777/
Messages:
{noformat}
This message was trimmed, see log for full details
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/spark-client/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp/conf
[copy] Copying 15 files to
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
spark-client ---
[INFO] Compiling 5 source files to
/data/hive-ptest/working/apache-github-source-source/spark-client/target/test-classes
[INFO]
[INFO] --- maven-dependency-plugin:2.8:copy (copy-guava-14) @ spark-client ---
[INFO] Configured Artifact: com.google.guava:guava:14.0.1:jar
[INFO] Copying guava-14.0.1.jar to
/data/hive-ptest/working/apache-github-source-source/spark-client/target/dependency/guava-14.0.1.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.16:test (default-test) @ spark-client ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.2:jar (default-jar) @ spark-client ---
[INFO] Building jar:
/data/hive-ptest/working/apache-github-source-source/spark-client/target/spark-client-2.2.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @
spark-client ---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @ spark-client ---
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/spark-client/target/spark-client-2.2.0-SNAPSHOT.jar
to
/data/hive-ptest/working/maven/org/apache/hive/spark-client/2.2.0-SNAPSHOT/spark-client-2.2.0-SNAPSHOT.jar
[INFO] Installing
/data/hive-ptest/working/apache-github-source-source/spark-client/pom.xml to
/data/hive-ptest/working/maven/org/apache/hive/spark-client/2.2.0-SNAPSHOT/spark-client-2.2.0-SNAPSHOT.pom
[INFO]
[INFO]
[INFO] Building Hive Query Language 2.2.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hive-exec ---
[INFO] Deleting /data/hive-ptest/working/apache-github-source-source/ql/target
[INFO] Deleting /data/hive-ptest/working/apache-github-source-source/ql
(includes = [datanucleus.log, derby.log], excludes = [])
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @
hive-exec ---
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (generate-sources) @ hive-exec ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/ql/target/generated-sources/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/ql/target/generated-sources/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/ql/target/generated-test-sources/java/org/apache/hadoop/hive/ql/exec/vector/expressions/gen
Generating vector expression code
Generating vector expression test code
[INFO] Executed tasks
[INFO]
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-source) @ hive-exec ---
[INFO] Source directory:
/data/hive-ptest/working/apache-github-source-source/ql/src/gen/thrift/gen-javabean
added.
[INFO] Source directory:
/data/hive-ptest/working/apache-github-source-source/ql/target/generated-sources/java
added.
[INFO]
[INFO] --- antlr3-maven-plugin:3.4:antlr (default) @ hive-exec ---
[INFO] ANTLR: Processing source directory
/data/hive-ptest/working/apache-github-source-source/ql/src/java
ANTLR Parser Generator Version 3.4
org/apache/hadoop/hive/ql/parse/HiveLexer.g
org/apache/hadoop/hive/ql/parse/HiveParser.g
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-exec ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hive-exec
---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[jira] [Commented] (HIVE-14342) Beeline output is garbled when executed from a remote shell
[
https://issues.apache.org/jira/browse/HIVE-14342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15408468#comment-15408468
]
Naveen Gangam commented on HIVE-14342:
--
After quite a bit of testing, I found that the most reliable means to detect if
beeline is being run via a remote shell is to check if the for the
process is a pipe.
So I have added an additional check to the condition before using jline's
UnsupportedTerminal. The use of UnsupportedTerminal should be limited to
beeline processes running locally as a background process.
The output now is as we expected prior to this change.
{code}
ssh -l root "sudo -u hive beeline -u
jdbc:hive2://localhost:1 -n hive -p hive --silent=true --outputformat=csv2
-f /tmp/run.sql"
root@'s password:
16/08/04 13:46:35 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
module jar containing PrefixTreeCodec is not present. Continuing without it.
key,name,value
1,TRUE,1
$
{code}
> Beeline output is garbled when executed from a remote shell
> ---
>
> Key: HIVE-14342
> URL: https://issues.apache.org/jira/browse/HIVE-14342
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 2.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>
> {code}
> use default;
> create table clitest (key int, name String, value String);
> insert into table clitest values
> (1,"TRUE","1"),(2,"TRUE","1"),(3,"TRUE","1"),(4,"TRUE","1"),(5,"FALSE","0"),(6,"FALSE","0"),(7,"FALSE","0");
> {code}
> then run a select query
> {code}
> # cat /tmp/select.sql
> set hive.execution.engine=mr;
> select key,name,value
> from clitest
> where value="1" limit 1;
> {code}
> Then run beeline via a remote shell, for example
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/select.sql"
> root@'s password:
> 16/07/12 14:59:22 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> nullkey,name,value
> 1,TRUE,1
> null
> $
> {code}
> In older releases that the output is as follows
> {code}
> $ ssh -l root "sudo -u hive beeline -u
> jdbc:hive2://localhost:1 -n hive -p hive --silent=true
> --outputformat=csv2 -f /tmp/run.sql"
> Are you sure you want to continue connecting (yes/no)? yes
> root@'s password:
> 16/07/12 14:57:55 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree
> module jar containing PrefixTreeCodec is not present. Continuing without it.
> key,name,value
> 1,TRUE,1
> $
> {code}
> The output contains nulls instead of blank lines. This is due to the use of
> -Djline.terminal=jline.UnsupportedTerminal introduced in HIVE-6758 to be able
> to run beeline as a background process. But this is the unfortunate side
> effect of that fix.
> Running beeline in background also produces garbled output.
> {code}
> # beeline -u "jdbc:hive2://localhost:1" -n hive -p hive --silent=true
> --outputformat=csv2 --showHeader=false -f /tmp/run.sql 2>&1 >
> /tmp/beeline.txt &
> # cat /tmp/beeline.txt
> null1,TRUE,1
> #
> {code}
> So I think the use of jline.UnsupportedTerminal should be documented but not
> used automatically by beeline under the covers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
