Hi, I'm trying out giraph with hive with instructions from http://giraph.apache.org/hive.html and I get the following error while reading data from hive
hadoop jar giraphstarter-jar-with-dependencies.jar org.apache.giraph.hive.HiveGiraphRunner -hiveconf hive.metastore.uris=thrift://..:50513 -hiveconf hive.metastore.sasl.enabled=true -hiveconf hive.metastore.kerberos.principal=hcat/... -db=dancehall -ei="com.yahoo.IdMappingsHiveToEdge, id_mappings, source=interstitials, dt=2015022323" -o "com.yahoo.IdClustersEdgeToHive, id_giraph, source=interstitials, dt=2015022323" --computationClass=com.yahoo.HiveConnectedComponents -w 1 15/02/27 04:18:58 INFO common.HiveMetastores: Connecting to metastore at ...50513 15/02/27 04:19:58 INFO common.BackoffRetryTask: Failed, but will retry in 25250 msec : org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out 15/02/27 04:20:24 INFO common.HiveMetastores: Connecting to metastore at ..:50513 15/02/27 04:21:24 INFO common.BackoffRetryTask: Failed, but will retry in 51984 msec : org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out 15/02/27 04:22:16 INFO common.HiveMetastores: Connecting to metastore at ...:50513 Exception in thread "main" java.lang.IllegalStateException: initialize: IOException occurred at com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:94) at org.apache.giraph.hive.input.edge.HiveEdgeInputFormat.setConf(HiveEdgeInputFormat.java:73) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.configureIfPossible(ImmutableClassesGiraphConfiguration.java:156) at org.apache.giraph.utils.ConfigurationUtils.configureIfPossible(ConfigurationUtils.java:151) at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:111) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:477) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createWrappedEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:488) at org.apache.giraph.hive.HiveGiraphRunner.prepareHiveEdgeInputs(HiveGiraphRunner.java:425) at org.apache.giraph.hive.HiveGiraphRunner.handleCommandLine(HiveGiraphRunner.java:615) at org.apache.giraph.hive.HiveGiraphRunner.run(HiveGiraphRunner.java:293) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.hive.HiveGiraphRunner.main(HiveGiraphRunner.java:286) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.io.IOException: No more retries left org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out at com.facebook.hiveio.common.BackoffRetryTask.run(BackoffRetryTask.java:111) at com.facebook.hiveio.schema.HiveTableSchemas.lookup(HiveTableSchemas.java:156) at com.facebook.hiveio.schema.HiveTableSchemas.initTableSchema(HiveTableSchemas.java:97) at com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:92) ... 18 more Any hints around this ? I work at Yahoo and here are versions we are using. Is Giraph and hive-io compatiable with these versions ? hadoop: 2.6.0.5.1501231242 Hive: 0.13.0.6.1502201219 -- Thanks, Kalyan
