Hi,
insert overwrite directory "$dir" select * from xxx;
creates files of type attempt_201008201925_165088_r_00_0.gz
insert overwrite table "$table" select * from xxx;
creates file of type attempt_201008201925_165088_r_00_0
How can I configure "insert overwrite directory" to pro
Hi,
insert overwrite directory "$dir" select * from xxx;
creates files of type attempt_201008201925_165088_r_00_0.gz
insert overwrite table "$table" select * from xxx;
creates file of type attempt_201008201925_165088_r_00_0
How can I configure "insert overwrite directory" to produ
add a phrase like "(Note:
only available starting with 0.7.0)" so that users of earlier Hive versions
know they need to upgrade if they want the feature.
> Be able to modify a partition's fileformat and fil
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL#Alter_Table.2BAC8-Partition_Location
This only change the metadata. With this patch, you will be able to let the
partition point to some external places, and use a new fileformat. If the
metadata you specified is correct, you will be able to do that.
>
ussions, its not clear to me what changes went
>in here.
It will be useful to summarize the use case which this jira satisfies. From
cursory look of the patch, it seems following is now possible to do
{code}
ALTER TABLE table_name [partitionSpec] SET LOCATION "new location" set
fileform
y a partition's fileformat and file location information.
> -
>
> Key: HIVE-1514
> URL: https://issues.apache.org/jira/browse/HIVE-1514
> Project: Hadoop Hive
>
modify a partition's fileformat and file location information.
> -
>
> Key: HIVE-1514
> URL: https://issues.apache.org/jira/browse/HIVE-1514
> Project: Hadoop Hive
[
https://issues.apache.org/jira/browse/HIVE-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-1514:
---
Attachment: hive-1514.3.patch
> Be able to modify a partition's fileformat and file
ILEFORMAT", "ALTERPARTITION_FILEFORMAR" });
There is a spelling mistake - should be
new String[] { "ALTERTABLE_FILEFORMAT", "ALTERPARTITION_FILEFORMAT" });
will result in changing a few log files.
> Be able to modify a parti
[
https://issues.apache.org/jira/browse/HIVE-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Namit Jain updated HIVE-1514:
-
Status: Open (was: Patch Available)
> Be able to modify a partition's fileformat and file
[
https://issues.apache.org/jira/browse/HIVE-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-1514:
---
Attachment: hive-1514.2.patch
> Be able to modify a partition's fileformat and file
[
https://issues.apache.org/jira/browse/HIVE-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-1514:
---
Status: Patch Available (was: Open)
> Be able to modify a partition's fileformat and file
[
https://issues.apache.org/jira/browse/HIVE-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-1514:
---
Attachment: hive-1514.1.patch
> Be able to modify a partition's fileformat and file
Be able to modify a partition's fileformat and file location information.
-
Key: HIVE-1514
URL: https://issues.apache.org/jira/browse/HIVE-1514
Project: Hadoop
that there is already a command to do the same thing:
ALTER TABLE table_name SET SERDE serde_class_name
http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL
> bug in 'set fileformat'
> ---
>
> Key: HIVE-1448
> URL:
[
https://issues.apache.org/jira/browse/HIVE-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-1448:
---
Attachment: hive-1448-brach-0.6.patch
hive-1448.1.patch
> bug in 'set fi
bug in 'set fileformat'
> ---
>
> Key: HIVE-1448
> URL: https://issues.apache.org/jira/browse/HIVE-1448
> Project: Hadoop Hive
> Issue Type: Bug
>Affects Versions: 0.6.0, 0.7.0
&
bug in 'set fileformat'
---
Key: HIVE-1448
URL: https://issues.apache.org/jira/browse/HIVE-1448
Project: Hadoop Hive
Issue Type: Bug
Affects Versions: 0.6.0, 0.7.0
Reporter: He
arSerde should not be the default Serde when user specified a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.
ult Serde when user specified a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.apache.org/jira/browse/HIVE-1085
&
Yongqiang!
> ColumnarSerde should not be the default Serde when user specified a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: ht
pass.
> ColumnarSerde should not be the default Serde when user specified a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.
patch is more
clear for future change.
> ColumnarSerde should not be the default Serde when user specified a
> fileformat using 'stored as'.
> -
>
> Key: H
d a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.apache.org/jira/browse/HIVE-1085
> Project: Hadoop
ser specified a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.apache.org/jira/browse/HIVE-1085
> Projec
d a
> fileformat using 'stored as'.
> -
>
> Key: HIVE-1085
> URL: https://issues.apache.org/jira/browse/HIVE-1085
> Project: Hadoop
the hive.default.fileformat is
set to RCFile. It should not ColumnarSerDe if the user specify a different
storage format in DDL.
> ColumnarSerde should not be the default Serde when user specified a
> fileformat using
ColumnarSerde should not be the default Serde when user specified a fileformat
using 'stored as'.
-
Key: HIVE-1085
URL: https://issues.apache.org/jira/b
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Shao updated HIVE-360:
Resolution: Fixed
Fix Version/s: 0.4.0
Release Note: HIVE-360. Generalize the FileFormat
case for that is inside text
file format. It's kind of rare so I think we can generalize that when we have a
second use case.
We should also move file format check function into the specific file format as
well.
> Generalize the FileFormat Inter
o me.
One comment is that the getFinalPath call should be made part of the
HiveOutputFormat as well. Actually all we need is to let the outputformat
determine the file extension. rest of the path name is always the same. but
it's not a big deal.
> Generalize the FileFormat Inter
neralize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
> Project: Hadoop Hive
> Issue Type: Improvement
&g
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12697408#action_12697408
]
Zheng Shao commented on HIVE-360:
-
+1
> Generalize the FileFormat Interface
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Attachment: hive-360-2009-04-09-3.patch
> Generalize the FileFormat Interface in H
oop.hive.ql.TestMTQueries FAILED
[junit] Test org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker
FAILED
[junit] Test org.apache.hadoop.hive.ql.parse.TestParse FAILED
"
fixed them and passed them in my local.
> Generalize
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Attachment: hive-360-2009-04-08-3.patch
> Generalize the FileFormat Interface in H
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Attachment: hive-360-2009-04-08.patch
> Generalize the FileFormat Interface in H
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Attachment: qfile.tar
> Generalize the FileFormat Interface in H
t on
OutputFormat.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
> Project: Hadoop Hive
> Issue Type:
differently.
6) modify tableDesc to support HiveOutputFormat directly
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
>
lose();
} catch (IOException e) {
fileIsSequenceFile = false;
}
{code}
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/j
d(jc)
+ Utilities.getFileExtension(jc, isCompressed));
}
{code}
Can you also make "initRecordWriter" in FileSinkOperator static? There is just
one member variable that is referenced: outPath, and you can make
initRecordWriter return the outWriter value.
> Generalize the FileFormat
java.util.Properties
2) added in two .q file. one in clientpositive and the other in clientnegative
Thanks Zheng.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/j
ABLE as well.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
> Project: Hadoop Hive
> Issue Type: Improvem
OUTPUTFORMAT xxx a good testcase
for OutputFormat.
Hive only uses OutputFormat in its FileSinkOperator, and it seems CREATE only
stores the information.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
>
x of CREATE TABLE ... STORED
AS INPUTFORMAT OUTPUTFORMAT xxx.
One positive test, and one negative test with invalid file formats (e.g., just
a hadoop file format, but no corresponding hive file format).
> Generalize the FileFormat Interface
iter method, can we change
it to java.util.Properties instead?
We don't need all the information about the table to create the File.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
>
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Status: Patch Available (was: Open)
> Generalize the FileFormat Interface in H
-03-32.patch.
It adds javadoc and the apache licence header in each new file.
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
>
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Status: Open (was: Patch Available)
> Generalize the FileFormat Interface in H
as its value. I think it is ok in Hive, isn't it?
Should i cancel the patch now and resubmit one once the refactory is done?
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: ht
submitted. That helps to get this on the radar for the
reviewers.
Thanks,
Ashish
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
>
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Thusoo reassigned HIVE-360:
--
Assignee: He Yongqiang
> Generalize the FileFormat Interface in H
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
> Project: Hadoop Hive
> Issue Type: Improvement
>
[
https://issues.apache.org/jira/browse/HIVE-360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
He Yongqiang updated HIVE-360:
--
Attachment: (was: hive-360-2009-03-31.patch)
> Generalize the FileFormat Interface in H
> Generalize the FileFormat Interface in Hive
> ---
>
> Key: HIVE-360
> URL: https://issues.apache.org/jira/browse/HIVE-360
> Project: Hadoop Hive
> Issue Type: Improvement
>
d in DDLSemanticAnalyzer.
{code}
case HiveParser.TOK_TBLSEQUENCEFILE:
inputFormat = SEQUENCEFILE_INPUT;
outputFormat = SEQUENCEFILE_OUTPUT;
break;
{code}
> Generalize the FileFormat Interface in Hive
> ---
>
>
Generalize the FileFormat Interface in Hive
---
Key: HIVE-360
URL: https://issues.apache.org/jira/browse/HIVE-360
Project: Hadoop Hive
Issue Type: Improvement
Reporter: Zheng Shao
58 matches
Mail list logo
