[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869738#comment-15869738
]
Steve Loughran commented on HADOOP-14076:
-
OR we have some utils.ConfIO class with methods like {{saveToFile(File,
Configuration) and saveToLocalPath(string, Configuration) as well as the same
for Filesystem URIs. It's just that Configuration is big enough as it is, and
these things don't have to be added to that class as it is. But yes, chris has
a point too: do it outside of the core and your app gets to use it immediately,
no need to backport
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866893#comment-15866893
]
Chris Douglas commented on HADOOP-14076:
I think Steve is suggesting that this belongs with the application code, not as
part of Hadoop.
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865940#comment-15865940
]
Ted Yu commented on HADOOP-14076:
-
Do you have suggestion where the helper method should reside.
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865739#comment-15865739
]
Steve Loughran commented on HADOOP-14076:
-
this is a very special case. Why not some helper java method
{{saveConfig(string path, Configuration}} somewhere?
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863813#comment-15863813
]
Ted Yu commented on HADOOP-14076:
-
That's what I did using JNI:
{code}
void writeConf(jobject conf, const char *filepath)
{
jclass class_fdesc = (*env)->FindClass(env, "java/io/FileDescriptor");
// construct a new FileDescriptor
jmethodID const_fdesc = (*env)->GetMethodID(env, class_fdesc, "",
"()V");
jobject file = (*env)->NewObject(env, class_fdesc, const_fdesc);
jfieldID field_fd = (*env)->GetFieldID(env, class_fdesc, "fd", "I");
int fd = open(filepath, O_RDWR | O_NONBLOCK | O_CREAT, S_IRWXU);
if (fd < 0) {
printf("Couldn't open file %s\n", filepath);
exit(-1);
}
(*env)->SetIntField(env, file, field_fd, fd);
jclass cls_outstream = (*env)->FindClass(env, "java/io/FileOutputStream");
jmethodID ctor_stream = (*env)->GetMethodID(env, cls_outstream, "",
"(Ljava/io/FileDescriptor;)V");
if (ctor_stream == NULL) {
printf("Couldn't get ctor for FileOutputStream\n");
exit(-1);
}
jobject file_outstream = (*env)->NewObject(env, cls_outstream, ctor_stream,
file);
if (file_outstream == NULL) {
printf("Couldn't create FileOutputStream\n");
exit(-1);
}
jclass class_conf = (*env)->FindClass(env, HADOOP_CONF);
jmethodID writeXmlMid = (*env)->GetMethodID(env, class_conf, "writeXml",
"(Ljava/io/OutputStream;)V");
(*env)->CallObjectMethod(env, conf, writeXmlMid, file_outstream);
}
{code}
The code is tedious (manipulating fd field).
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14076) Allow Configuration to be persisted given path to file
[
https://issues.apache.org/jira/browse/HADOOP-14076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863804#comment-15863804
]
Steve Loughran commented on HADOOP-14076:
-
Ted, what's wrong with just creating the FileOutputStream and calling writeXML?
It's not that hard
> Allow Configuration to be persisted given path to file
> --
>
> Key: HADOOP-14076
> URL: https://issues.apache.org/jira/browse/HADOOP-14076
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ted Yu
>
> Currently Configuration has the following methods for persistence:
> {code}
> public void writeXml(OutputStream out) throws IOException {
> public void writeXml(Writer out) throws IOException {
> {code}
> Adding API for persisting to file given path would be useful:
> {code}
> public void writeXml(String path) throws IOException {
> {code}
> Background: I recently worked on exporting Configuration to a file using JNI.
> Without the proposed API, I resorted to some trick such as the following:
> http://www.kfu.com/~nsayer/Java/jni-filedesc.html
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
