[jira] [Comment Edited] (PHOENIX-4056) java.lang.IllegalArgumentException: Can not create a Path from an empty string
[
https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358903#comment-16358903
]
Josh Mahonin edited comment on PHOENIX-4056 at 2/9/18 8:19 PM:
---
Pushed to:
* master
* 4.x-cdh5.11.2
* 4.x-HBase-0.98
* 4.x-HBase-1.2
* 4.x-HBase-1.3
* 5.x-HBase-2.0
Thanks for the contribution [~stepson] !
was (Author: jmahonin):
Pushed to:
- master
- 4.x-cdh5.11.2
- 4.x-HBase-0.98
- 4.x-HBase-1.2
- 4.x-HBase-1.3
- 5.x-HBase-2.0
Thanks for the contribution [~stepson] !
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> --
>
> Key: PHOENIX-4056
> URL: https://issues.apache.org/jira/browse/PHOENIX-4056
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
> Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
>Reporter: Jepson
>Priority: Major
> Labels: features, patch, test
> Attachments: PHOENIX-4056.patch, PHOENIX-4056_v2.patch,
> PHOENIX-4056_v3.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
>
> phoenix.schema.isNamespaceMappingEnabled
> true
>
>
> phoenix.schema.mapSystemTablesToNamespace
> true
>
> 2.The Code:
> {code:java}
> resultDF.write
> .format("org.apache.phoenix.spark")
> .mode(SaveMode.Overwrite)
> .option("table", "JYDW.ADDRESS_ORDERCOUNT")
> .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
> .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at
> SparkHadoopMapReduceWriter.scala:88, took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job
> job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path
> from an empty string*{color}
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> at
> org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:994)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:59)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:28)
> at
> org.apache.phoenix.spark.DefaultSource.createRelation(DefaultSource.scala:47)
> at
> org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:472)
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:48)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at
>
[jira] [Comment Edited] (PHOENIX-4056) java.lang.IllegalArgumentException: Can not create a Path from an empty string
[
https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358768#comment-16358768
]
James Taylor edited comment on PHOENIX-4056 at 2/9/18 5:58 PM:
---
Thanks, [~jmahonin]! Yes, those are the right branches. The CDH one too,
though.
was (Author: jamestaylor):
Thanks, [~jmahonin]! Yes, those are the right branches.
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> --
>
> Key: PHOENIX-4056
> URL: https://issues.apache.org/jira/browse/PHOENIX-4056
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
> Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
>Reporter: Jepson
>Priority: Major
> Labels: features, patch, test
> Attachments: PHOENIX-4056.patch, PHOENIX-4056_v2.patch,
> PHOENIX-4056_v3.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
>
> phoenix.schema.isNamespaceMappingEnabled
> true
>
>
> phoenix.schema.mapSystemTablesToNamespace
> true
>
> 2.The Code:
> {code:java}
> resultDF.write
> .format("org.apache.phoenix.spark")
> .mode(SaveMode.Overwrite)
> .option("table", "JYDW.ADDRESS_ORDERCOUNT")
> .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
> .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at
> SparkHadoopMapReduceWriter.scala:88, took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job
> job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path
> from an empty string*{color}
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> at
> org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:994)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:59)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:28)
> at
> org.apache.phoenix.spark.DefaultSource.createRelation(DefaultSource.scala:47)
> at
> org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:472)
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:48)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> at
>
[jira] [Comment Edited] (PHOENIX-4056) java.lang.IllegalArgumentException: Can not create a Path from an empty string
[
https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355485#comment-16355485
]
Josh Mahonin edited comment on PHOENIX-4056 at 2/7/18 2:11 PM:
---
I don't have much time to do more than a cursory review right now,
unfortunately. Overall I like the approach, but:
a) What happens if 'mapred.output.dir' isn't set? Or is that always the case?
org.apache.hadoop.conf.Configuration::get() will return a null in that case. Is
Spark OK with that? Maybe another idea is to wrap it up in a scala Option,
something like:
{{Option(conf.get("mapred.output.dir")).getOrElse("")}}
a) Any ideas on if this is backwards compatible down to Spark 2.0? It maybe an
impossible goal, but it would be nice to stay binary compatible with the whole
2.x range, if possible.
was (Author: jmahonin):
I don't have much time to do more than a cursory review right now,
unfortunately. Overall I like the approach, but:
a) What happens if 'mapred.output.dir' isn't set? Or is that always the case?
org.apache.hadoop.conf.Configuration::get() will return a null in that case. Is
Spark OK with that? Maybe another idea is to wrap it up in a scala Option,
something like:
{{Option(conf.get('mapred.output.dir')).getOrElse("")}}
a) Any ideas on if this is backwards compatible down to Spark 2.0? It maybe an
impossible goal, but it would be nice to stay binary compatible with the whole
2.x range, if possible.
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> --
>
> Key: PHOENIX-4056
> URL: https://issues.apache.org/jira/browse/PHOENIX-4056
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
> Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
>Reporter: Jepson
>Priority: Major
> Labels: features, patch, test
> Attachments: PHOENIX-4056.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
>
> phoenix.schema.isNamespaceMappingEnabled
> true
>
>
> phoenix.schema.mapSystemTablesToNamespace
> true
>
> 2.The Code:
> {code:java}
> resultDF.write
> .format("org.apache.phoenix.spark")
> .mode(SaveMode.Overwrite)
> .option("table", "JYDW.ADDRESS_ORDERCOUNT")
> .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
> .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at
> SparkHadoopMapReduceWriter.scala:88, took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job
> job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path
> from an empty string*{color}
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> at
> org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
>
[jira] [Comment Edited] (PHOENIX-4056) java.lang.IllegalArgumentException: Can not create a Path from an empty string
[
https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112202#comment-16112202
]
Jepson edited comment on PHOENIX-4056 at 8/3/17 5:16 AM:
-
I degradate the spark version from 2.2.0 to 2.1.1, is resolved with the error.
So phoenix 4.11 -hbase1.20 with spark2.2.0 is not work, the compatibility is
not good.
was (Author: 1028344...@qq.com):
I degradate the spark version from 2.2.0 to 2.1.1, is resolved with the error.
> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> --
>
> Key: PHOENIX-4056
> URL: https://issues.apache.org/jira/browse/PHOENIX-4056
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
> Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
>Reporter: Jepson
> Labels: features, patch, test
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
>
> phoenix.schema.isNamespaceMappingEnabled
> true
>
>
> phoenix.schema.mapSystemTablesToNamespace
> true
>
> 2.The Code:
> {code:java}
> resultDF.write
> .format("org.apache.phoenix.spark")
> .mode(SaveMode.Overwrite)
> .option("table", "JYDW.ADDRESS_ORDERCOUNT")
> .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
> .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at
> SparkHadoopMapReduceWriter.scala:88, took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job
> job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path
> from an empty string*{color}
> at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> at org.apache.hadoop.fs.Path.(Path.java:134)
> at org.apache.hadoop.fs.Path.(Path.java:88)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> at
> org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> at
> org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> at
> org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:994)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:59)
> at
> org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:28)
> at
> org.apache.phoenix.spark.DefaultSource.createRelation(DefaultSource.scala:47)
> at
> org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:472)
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:48)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> at
>
