[
https://issues.apache.org/jira/browse/HIVE-20447?focusedWorklogId=456176&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-456176
]
ASF GitHub Bot logged work on HIVE-20447:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Jul/20 14:49
Start Date: 08/Jul/20 14:49
Worklog Time Spent: 10m
Work Description: belugabehr commented on a change in pull request #1169:
URL: https://github.com/apache/hive/pull/1169#discussion_r451603702
##########
File path: beeline/src/java/org/apache/hive/beeline/JSONOutputFormat.java
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.9
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.sql.SQLException;
+import java.sql.Types;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+
+import com.fasterxml.jackson.core.JsonEncoding;
+import com.fasterxml.jackson.core.JsonFactory;
+import com.fasterxml.jackson.core.JsonGenerator;
+
+/**
+ * OutputFormat for standard JSON format.
+ *
+ */
+public class JSONOutputFormat extends AbstractOutputFormat {
+ protected final BeeLine beeLine;
+ protected JsonGenerator generator;
+
+
+ /**
+ * @param beeLine
+ */
+ JSONOutputFormat(BeeLine beeLine){
+ this.beeLine = beeLine;
+ ByteArrayOutputStream buf = new ByteArrayOutputStream();
+ try{
+ this.generator = new JsonFactory().createGenerator(buf,
JsonEncoding.UTF8);
+ }catch(IOException e){
+ beeLine.handleException(e);
+ }
+ }
+
+ @Override
+ void printHeader(Rows.Row header) {
+ try {
+ generator.writeStartObject();
+ generator.writeArrayFieldStart("resultset");
+ } catch (IOException e) {
+ beeLine.handleException(e);
+ }
+ }
+
+ @Override
+ void printFooter(Rows.Row header) {
+ try {
+ generator.writeEndArray();
+ generator.writeEndObject();
+ beeLine.output(generator.getOutputTarget().toString());
+ generator.flush();
Review comment:
Typically you want to `flush` the object to the underlying stream
(`ByteArrayOutputStream` in this case) in case the object is doing any kind of
internal buffering in order to "flush" its content out.
While your `beeLine.output` is technically correct. It's a bit confusing to
other coders.
Order of operations should be:
1. Flush the generator to clear any buffering into the target `OutputStream`
2. Convert the `OutputStream` into the target output format (String in this
situation)
3. Please use a `new String()` with UTF-8 encoding explicitly specified here
https://docs.oracle.com/javase/8/docs/api/java/lang/String.html#String-byte:A-java.nio.charset.Charset-
##########
File path: beeline/src/java/org/apache/hive/beeline/JSONOutputFormat.java
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.9
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.sql.SQLException;
+import java.sql.Types;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+
+import com.fasterxml.jackson.core.JsonEncoding;
+import com.fasterxml.jackson.core.JsonFactory;
+import com.fasterxml.jackson.core.JsonGenerator;
+
+/**
+ * OutputFormat for standard JSON format.
+ *
+ */
+public class JSONOutputFormat extends AbstractOutputFormat {
+ protected final BeeLine beeLine;
+ protected JsonGenerator generator;
+
+
+ /**
+ * @param beeLine
+ */
+ JSONOutputFormat(BeeLine beeLine){
+ this.beeLine = beeLine;
+ ByteArrayOutputStream buf = new ByteArrayOutputStream();
+ try{
+ this.generator = new JsonFactory().createGenerator(buf,
JsonEncoding.UTF8);
+ }catch(IOException e){
+ beeLine.handleException(e);
+ }
+ }
+
+ @Override
+ void printHeader(Rows.Row header) {
+ try {
+ generator.writeStartObject();
+ generator.writeArrayFieldStart("resultset");
+ } catch (IOException e) {
+ beeLine.handleException(e);
+ }
+ }
+
+ @Override
+ void printFooter(Rows.Row header) {
+ try {
+ generator.writeEndArray();
+ generator.writeEndObject();
+ beeLine.output(generator.getOutputTarget().toString());
+ generator.flush();
+ } catch (IOException e) {
+ beeLine.handleException(e);
+ }
+ }
+
+ @Override
+ void printRow(Rows rows, Rows.Row header, Rows.Row row) {
Review comment:
I'm a bit surprised by this. Why does this not write a
`startObject`/`endObject` per row?
##########
File path: beeline/src/java/org/apache/hive/beeline/JSONOutputFormat.java
##########
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.9
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.sql.SQLException;
+import java.sql.Types;
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+
+import com.fasterxml.jackson.core.JsonEncoding;
+import com.fasterxml.jackson.core.JsonFactory;
+import com.fasterxml.jackson.core.JsonGenerator;
+
+/**
+ * OutputFormat for standard JSON format.
+ *
+ */
+public class JSONOutputFormat extends AbstractOutputFormat {
+ protected final BeeLine beeLine;
+ protected JsonGenerator generator;
+
+
+ /**
+ * @param beeLine
+ */
+ JSONOutputFormat(BeeLine beeLine){
+ this.beeLine = beeLine;
+ ByteArrayOutputStream buf = new ByteArrayOutputStream();
+ try{
+ this.generator = new JsonFactory().createGenerator(buf,
JsonEncoding.UTF8);
+ }catch(IOException e){
Review comment:
Nit: formatting issue
##########
File path: beeline/src/java/org/apache/hive/beeline/JSONFileOutputFormat.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.9
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.io.IOException;
+
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+
+/**
+ * OutputFormat for hive JSON file format
+ * Removes "{ "resultset": [...] }" wrapping and prints one object per line
+ *
+ */
+public class JSONFileOutputFormat extends JSONOutputFormat {
+
+
+ JSONFileOutputFormat(BeeLine beeLine) {
+ super(beeLine);
+ this.generator.setPrettyPrinter(new MinimalPrettyPrinter("\n"));
+ }
+
+ @Override
+ void printHeader(Rows.Row header) {}
+
+ @Override
+ void printFooter(Rows.Row header) {}
+
+ @Override
+ void printRow(Rows rows, Rows.Row header, Rows.Row row) {
+ try {
+ this.generator.writeStartObject();
+ super.printRow(rows, header, row);
+ this.generator.writeEndObject();
+ } catch (IOException e) {
+ this.beeLine.handleException(e);
+ }
+ }
+}
Review comment:
Add newline to end of files :)
##########
File path: beeline/src/main/resources/BeeLine.properties
##########
@@ -194,7 +194,7 @@ cmd-usage: Usage: java org.apache.hive.cli.beeline.BeeLine
\n \
\ --silent=[true/false] be more silent\n \
\ --report=[true/false] show number of rows and execution time
after query execution\n \
\ --autosave=[true/false] automatically save preferences\n \
-\ --outputformat=[table/vertical/csv2/tsv2/dsv/csv/tsv] format mode for
result display\n \
+\ --outputformat=[table/vertical/csv2/tsv2/dsv/csv/tsv/json] format mode for
result display\n \
Review comment:
Needs `jsonfile` as well
##########
File path: beeline/src/java/org/apache/hive/beeline/JSONFileOutputFormat.java
##########
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * This source file is based on code taken from SQLLine 1.9
+ * See SQLLine notice in LICENSE
+ */
+package org.apache.hive.beeline;
+
+import java.io.IOException;
+
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+
+/**
+ * OutputFormat for hive JSON file format
+ * Removes "{ "resultset": [...] }" wrapping and prints one object per line
+ *
Review comment:
Nit: The Hive style guidelines require a period at the end of the first
sentence here.
Please provide an example and provide some additional context... something
like...
```
This output format matches the same format as a Hive table created with
JSONFILE file format:
CREATE TABLE ... STORED AS JSONFILE;
```
Then provide a sample output in the JavaDoc. Feel free to use HTML tags and
JavaDoc docs to format and enhance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 456176)
Time Spent: 1h 50m (was: 1h 40m)
> Add JSON Outputformat support
> -----------------------------
>
> Key: HIVE-20447
> URL: https://issues.apache.org/jira/browse/HIVE-20447
> Project: Hive
> Issue Type: Task
> Components: Beeline
> Reporter: Max Efremov
> Assignee: Hunter Logan
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20447.01.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This function is present in SQLLine. We need add it to beeline too.
> https://github.com/julianhyde/sqlline/pull/84
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
