Hi all, I'm trying to run a MapReduce job fetching data from the Datastore and outputting into BigQuery following the steps here: https://developers.google.com/hadoop/datastore-to-bigquery-sample#samplecode
I've got the Datastore and project all set up, as well as Bigquery, the
Hadoop cluster running, but while running both the job I wrote and the
sample "test-mr-datastore-to-bigquery.sh", I'm getting a DatastoreException
error:
hadoop-m:tCaused by:
com.google.api.services.datastore.client.DatastoreException: a composite
filter must have at least one sub-filter
hadoop-m:t at
com.google.api.services.datastore.client.RemoteRpc.makeException(RemoteRpc.java:114)
hadoop-m:t at
com.google.api.services.datastore.client.RemoteRpc.call(RemoteRpc.java:80)
hadoop-m:t at
com.google.api.services.datastore.client.Datastore.runQuery(Datastore.java:109)
hadoop-m:t at
com.google.cloud.hadoop.io.datastore.DatastoreIterator.getIteratorAndMoveCursor(DatastoreIterator.java:67)
hadoop-m:t ... 12 more
hadoop-m:tCaused by: com.google.api.client.http.HttpResponseException: 400
Bad Request
hadoop-m:ta composite filter must have at least one sub-filter
hadoop-m:t at
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1054)
hadoop-m:t at
com.google.api.services.datastore.client.RemoteRpc.call(RemoteRpc.java:77)
hadoop-m:t ... 14 more
Full log is attached.
On my job, the Exception specify the full query:
14/09/04 23:46:56 INFO mapred.JobClient: Task Id :
attempt_201409042325_0001_m_000000_2, Status : FAILED
java.io.IOException: Error connecting to the Datastore to run query: kind {
name: "FacebookProfile"
}
filter {
composite_filter {
operator: AND
}
}
ERROR:a composite filter must have at least one sub-filter
but the query I'm generating doesn't have any filtering:
Query.Builder q = Query.newBuilder();
KindExpression.Builder kind = KindExpression.newBuilder();
kind.setName(inputKindName);
q.addKind(kind);
String query = TextFormat.printToString(q);
DatastoreHadoopInputFormat.setInput(job, query, datastoreDatasetId);
Extra info:
Bdutil version: bdutil-0.35.1
For the sample, I'm using everything that is contained within the samples
folder, and for my Job I also used these two jars:
bigquery-connector-0.4.3
datastore-connector-0.14.6
is there any Maven repository where to access them?
Thanks and regards,
Andrea
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.
hadoop-sample.log
Description: Binary data
