On Mon, Oct 25, 2010 at 1:42 PM, lei liu <liulei...@gmail.com> wrote:
> Hi Edward Capriolo,
> Thank you for your replay.
>
> I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command , if
> there is the directory in HDFS, the JDBC don't throw Exception, if there is
> not the directory in HDFS, the JDBC will throw Exception. Could you tell I
> can use the way to judge whether the directory is in HDFS?
>
>
> Thanks.
>
> 2010/10/25 Edward Capriolo <edlinuxg...@gmail.com>
>>
>> On Mon, Oct 25, 2010 at 9:04 AM, lei liu <liulei...@gmail.com> wrote:
>> > I use JDBC to run "dfs -lsr hdfs://hdpnn:9000/group/hive/" command, but
>> > ResultSet don't return value, the code like below:
>> >
>> > Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver");
>> > hiveConnection =
>> > DriverManager.getConnection("jdbc:hive://10.232.11.53:10000/default" ,
>> > "root", "root");
>> > statement = hiveConnection.createStatement();
>> > String sql = "dfs -mkdir
>> > hdfs://hdpnn:9000//group/tbprofile/mianfeng/scratch/test/001/ ";
>> >
>> > ResultSet result = statement.executeQuery(sql); //the
>> > ResultSet
>> > don't return any value.
>> > while(result.next()) {
>> >
>> > System.out.println("day_id =" + result.getString(1)
>> > );//the
>> > }
>> >
>> > statement.execute("set remove.scratch.dir=ture");
>> > statement.close();
>> >
>> >
>> > I run the command in CLI, there is output content, the content like
>> > below
>> > :
>> > drwxr-x--- - mianfeng cug-tbprofile 0 2010-10-25 20:55
>> > /group/tbprofile/mianfeng/scratch/test/001
>> >
>> >
>> >
>>
>> dfs and ! statements are handled specially in the CLI. They are not
>> run through the Query Processor and do not produce a result set.
>
>
You could suggest possible extensions to hive language to deal with
these type of things.
>From what you are sounds like the existence of a directory causes an
exception to be thrown. You could trap the exeption using try catch.
One other idea comes to mind. Hive has a duel "virtual table" that
allows you do do things like
select 1+2 from duel. You could design a UDF that tests for existence of a file.
select directoryExists("/user/bla") from duel, then use this in your workflow.
Interesting idea. :) If you do not make a directoryExists udf, i might :)
