[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.7.patch Attached a new patch with cleaner negotiation and support for versioned plug-ins. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.6.patch, HIVE-13680.7.patch, HIVE-13680.patch, > proposal.pdf, SnappyCompDe.zip > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15384) Compressor plugin
[ https://issues.apache.org/jira/browse/HIVE-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-15384: -- Description: Splitting compressor into separate JIRA from server framework (was: Splitting server framework into separate JIRA from compressor) > Compressor plugin > - > > Key: HIVE-15384 > URL: https://issues.apache.org/jira/browse/HIVE-15384 > Project: Hive > Issue Type: Sub-task >Reporter: Ziyang Zhao >Assignee: Kevin Liew > > Splitting compressor into separate JIRA from server framework -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15384) Snappy Compressor plugin
[ https://issues.apache.org/jira/browse/HIVE-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-15384: -- Summary: Snappy Compressor plugin (was: Compressor plugin) > Snappy Compressor plugin > > > Key: HIVE-15384 > URL: https://issues.apache.org/jira/browse/HIVE-15384 > Project: Hive > Issue Type: Sub-task >Reporter: Ziyang Zhao >Assignee: Kevin Liew > > Splitting compressor into separate JIRA from server framework -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-15384) Compressor plugin
[ https://issues.apache.org/jira/browse/HIVE-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew reassigned HIVE-15384: - Assignee: Kevin Liew > Compressor plugin > - > > Key: HIVE-15384 > URL: https://issues.apache.org/jira/browse/HIVE-15384 > Project: Hive > Issue Type: Sub-task >Reporter: Ziyang Zhao >Assignee: Kevin Liew > > Splitting server framework into separate JIRA from compressor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.6.patch Latest patch attached. The server-side framework will remain in this JIRA while the sample compressor moves to HIVE-15384. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.6.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-15384) Compressor plugin
[ https://issues.apache.org/jira/browse/HIVE-15384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-15384: -- Summary: Compressor plugin (was: Compressor plugin framework) > Compressor plugin > - > > Key: HIVE-15384 > URL: https://issues.apache.org/jira/browse/HIVE-15384 > Project: Hive > Issue Type: Sub-task >Reporter: Ziyang Zhao > > Splitting server framework into separate JIRA from compressor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13830) Hive on spark driver crash with Spark 1.6.1
[
https://issues.apache.org/jira/browse/HIVE-13830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Liew updated HIVE-13830:
--
Description:
With Hive 1.2.1 I was able to use Hive on successfully with the use of the
-assembly "-assembly-1.4.1-hadoop2.7.1.jar".
Today with Hive 2.0.0, I'm unable to use Hive on whether it be with the
-assembly "-assembly-1.4.1-hadoop2.7.1.jar" or the -assembly
"-assembly-1.6.1-hadoop2.7.2.jar".
My configuration is the following:
* -. available in HIVE_DIR/
* assembly available in HIVE_DIR/lib
I gathered several logs below:
- HQL commands
{noformat}
$ hive -v --database shfs3453
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/opt/application/Hive/apache-hive-2.0.0-bin/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/application/Hive/apache-hive-2.0.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/application//-1.6.1/assembly/target/scala-2.10/-assembly-1.6.1-hadoop2.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in
file:/opt/application/Hive/apache-hive-2.0.0-bin/conf/hive-log4j2.properties
use shfs3453
OK
Time taken: 1.425 seconds
Hive-on-MR is deprecated in Hive 2 and may not be available in the future
versions. Consider using a different execution engine (i.e. tez, ) or using
Hive 1.X releases.
hive (shfs3453)> set hive.execution.engine=;
set hive.execution.engine=
hive (shfs3453)> set .master=yarn-client;
set .master=yarn-client
hive (shfs3453)> CREATE TABLE chicagoCrimes2 (ID BIGINT, CaseNumber STRING, Day
STRING, Block STRING, IUCR INT, PrimaryType STRING, Description STRING,
LocationDescription STRING, Arrest BOOLEAN, Domestic BOOLEAN, Beat INT,
District INT, Ward INT, CommunityArea INT, FBICode INT, XCoordinate BIGINT,
YCoordinate BIGINT, Year INT, UpdatedOn STRING, Latitude FLOAT, Longitude
FLOAT, Location STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS
TEXTFILE;
CREATE TABLE chicagoCrimes2 (ID BIGINT, CaseNumber STRING, Day STRING, Block
STRING, IUCR INT, PrimaryType STRING, Description STRING, LocationDescription
STRING, Arrest BOOLEAN, Domestic BOOLEAN, Beat INT, District INT, Ward INT,
CommunityArea INT, FBICode INT, XCoordinate BIGINT, YCoordinate BIGINT, Year
INT, UpdatedOn STRING, Latitude FLOAT, Longitude FLOAT, Location STRING) ROW
FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE
OK
Time taken: 0.408 seconds
hive (shfs3453)> INSERT OVERWRITE TABLE chicagocrimes2 SELECT * FROM
chicagocrimes WHERE Description = 'FIRST DEGREE MURDER';
INSERT OVERWRITE TABLE chicagocrimes2 SELECT * FROM chicagocrimes WHERE
Description = 'FIRST DEGREE MURDER'
Query ID = shfs3453_20160524092714_41c89aec-2c6f-49e9-98c7-d227ca144f73
Total jobs = 1
Launching Job 1 out of 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=
In order to set a constant number of reducers:
set mapreduce.job.reduces=
Starting Job = 79484279-8e75-4b13-8e71-7de463f4d51e
Status: SENT
Failed to execute task, with exception 'java.lang.IllegalStateException(RPC
channel is closed.)'
FAILED: Execution , return code 1 from org.apache.hadoop.hive.ql.exec..SparkTask
{noformat}
- Client logs
{noformat}
May 24 09:32:19 hive-cli - org.apache.hive..client.rpc.RpcDispatcherReceived
message:io.netty.handler.codec.DecoderException:
java.lang.NoClassDefFoundError: org/apache/hive//client/Job
at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:358)
at
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:230)
at
io.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: SnappyCompDe.zip > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: SnappyCompDe.zip) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.patch, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Patch Available (was: Open) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Open (was: Patch Available) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.4.patch > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.4.patch, HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.3.patch > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.3.patch, > HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: SnappyCompDe.zip) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: SnappyCompDe.zip > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425160#comment-15425160 ] Kevin Liew commented on HIVE-13680: --- I updated the Docker image and SnappyCompDe to reflect changes to the CompDe interface > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Patch Available (was: Open) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.2.patch > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: HIVE-13680.2.patch) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Open (was: Patch Available) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/17/16 3:06 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip,
> proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/16/16 11:00 PM:
-
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
EDIT: the Docker image is outdated. I will update it later
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip,
> proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: SnappyCompDe.zip) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: SnappyCompDe.zip > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/16/16 10:48 PM:
-
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
EDIT: the Docker image is outdated. I will update it later
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
EDIT: the Docker image is outdated. I will update it later
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip,
> proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/16/16 10:47 PM:
-
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
EDIT: the Docker image is outdated. I will update it later
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip,
> proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Patch Available (was: Open) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Open (was: Patch Available) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.2.patch Fixed an issue where the default value of a new ConfVar was null instead of empty string. Added a missing `isSet~` check for Thrift message fields. Set the default CompDe to none because unit tests do not have the plug-in jar. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.2.patch, HIVE-13680.patch, SnappyCompDe.zip, > proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 9:09 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
Alternatively you can use the Docker image that I have been using for
development: https://github.com/kliewkliew/docker-hive-dev/tree/HIVE-13680
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:16 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:12 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:12 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf hive.server2.thrift.resultset.compressor.list=.{noformat}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:11 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf hive.server2.thrift.resultset.compressor.list=.{code}
was (Author: kliew):
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{code}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:12 PM:
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{noformat}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{noformat}
was (Author: kliew):
Example compressor attached. Configure the server with
{code:xml}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf hive.server2.thrift.resultset.compressor.list=.{code}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:11 PM:
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{code}
was (Author: kliew):
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf hive.server2.thrift.resultset.compressor.list=.{code}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:11 PM:
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
.
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
, add the example CompDe to the Hive lib folder,
and start beeline with
{code}--hiveconf hive.server2.thrift.resultset.compressor.list=.{code}
was (Author: kliew):
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
and start beeline with
{code}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{code}
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[
https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421390#comment-15421390
]
Kevin Liew edited comment on HIVE-13680 at 8/15/16 6:10 PM:
Example compressor attached. Configure the server with
{code}
hive.server2.thrift.resultset.server.compressor.list
snappy.snappy
hive.server2.thrift.resultset.serialize.in.tasks
true
{code}
and start beeline with
{code}--hiveconf
hive.server2.thrift.resultset.compressor.list=snappy.snappy{code}
was (Author: kliew):
Example Snappy compressor attached.
> HiveServer2: Provide a way to compress ResultSets
> -
>
> Key: HIVE-13680
> URL: https://issues.apache.org/jira/browse/HIVE-13680
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, JDBC
>Reporter: Vaibhav Gumashta
>Assignee: Kevin Liew
> Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf
>
>
> With HIVE-12049 in, we can provide an option to compress ResultSets before
> writing to disk. The user can specify a compression library via a config
> param which can be used in the tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: SnappyCompDe.zip Example Snappy compressor attached. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.patch, SnappyCompDe.zip, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Status: Patch Available (was: In Progress) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.patch, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Work started] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-13680 started by Kevin Liew. - > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.patch, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: HIVE-13680.patch First patch submitted. HiveConf settings submitted by the client are required to be prefixed by "set:hiveconf" so in ThriftCLIService we have to preserve this prefix to ensure that the SessionState is generated correctly. To make the code cleaner, we could expose functions (which are currently out of scope) to parse the prefix, or we could have the client send the list of compressors and list of configs in new fields in the Thrift message (after compressor negotiation, the CompDe would be stored in a new field in SessionState instead of in SessionState.sessConf). I prefer the second option. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: HIVE-13680.patch, proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15377135#comment-15377135 ] Kevin Liew commented on HIVE-13680: --- I attached an updated proposal. I think we can forward with this but additional feedback is welcome. [~alangates] I looked more into the ORC format. ORC files are optimized through indexes and metadata for retrieving subsets of stored data. When serving remote clients: we have already extracted the final result-set, so ORC metadata is just overhead. ORC also does not provide plugglable compressors. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285570#comment-15285570 ] Kevin Liew commented on HIVE-13680: --- Ah, that makes more sense. I was apprehensive of that approach which is why I researched and included points about security options. I'll revise the document. > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: (was: proposal.pdf) > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew updated HIVE-13680: -- Attachment: proposal.pdf Hi Vaibhav, I've attached a proposal draft with your suggestion of having the client send the compressor plugin. I've finished the high-level overview and started getting into the implementation details. I’m also maintaining a document that we can use as a starting point for discussion. https://github.com/kliewkliew/HIVE-13680/blob/master/design-considerations/design-considerations.pdf > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > Attachments: proposal.pdf > > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-13680) HiveServer2: Provide a way to compress ResultSets
[ https://issues.apache.org/jira/browse/HIVE-13680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew reassigned HIVE-13680: - Assignee: Kevin Liew > HiveServer2: Provide a way to compress ResultSets > - > > Key: HIVE-13680 > URL: https://issues.apache.org/jira/browse/HIVE-13680 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, JDBC >Reporter: Vaibhav Gumashta >Assignee: Kevin Liew > > With HIVE-12049 in, we can provide an option to compress ResultSets before > writing to disk. The user can specify a compression library via a config > param which can be used in the tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HIVE-10438) HiveServer2: Enable ResultSet compression for HS2/*DBC driver
[ https://issues.apache.org/jira/browse/HIVE-10438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Liew reassigned HIVE-10438: - Assignee: Kevin Liew (was: Rohit Dholakia) > HiveServer2: Enable ResultSet compression for HS2/*DBC driver > - > > Key: HIVE-10438 > URL: https://issues.apache.org/jira/browse/HIVE-10438 > Project: Hive > Issue Type: Sub-task > Components: Hive, Thrift API >Affects Versions: 1.2.0 >Reporter: Rohit Dholakia >Assignee: Kevin Liew > Labels: patch > Attachments: HIVE-10438-1.patch, HIVE-10438.patch, > Proposal-rscompressor.pdf, README.txt, > Results_Snappy_protobuf_TBinary_TCompact.pdf, hs2ResultSetCompressor.zip, > hs2driver-master.zip > > > This JIRA proposes an architecture for enabling ResultSet compression which > uses an external plugin. > The patch has three aspects to it: > 0. An architecture for enabling ResultSet compression with external plugins > 1. An example plugin to demonstrate end-to-end functionality > 2. A container to allow everyone to write and test ResultSet compressors with > a query submitter (https://github.com/xiaom/hs2driver) > Also attaching a design document explaining the changes, experimental results > document, and a pdf explaining how to setup the docker container to observe > end-to-end functionality of ResultSet compression. > https://reviews.apache.org/r/35792/ Review board link. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
