[
https://issues.apache.org/jira/browse/BEAM-5966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles resolved BEAM-5966.
-----------------------------------
Resolution: Not A Problem
Fix Version/s: Not applicable
> When working "LiteralGqlQuery", Error: Query cannot have any sort orders.
> -------------------------------------------------------------------------
>
> Key: BEAM-5966
> URL: https://issues.apache.org/jira/browse/BEAM-5966
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.8.0
> Reporter: Murat ODUNC
> Assignee: Chamikara Jayalath
> Priority: Major
> Fix For: Not applicable
>
>
> Hello..
> I have used Apache Beam pipelines on my project. And I have a problem to read
> data from GCP DataStore.
>
> My First question is "How to create "Query" object" . I'am sharing a sample
> of regarding JavaDoc.
> [https://beam.apache.org/releases/javadoc/2.8.0/]
> {code:java}
> Query query = ...; // I dont know any idea how configure the object
> String projectId = "...";
> Pipeline p = Pipeline.create(options);
> PCollection<Entity> entities = p.apply(
> DatastoreIO.v1().read()
> .withProjectId(projectId)
> .withQuery(query));
> {code}
> My second question is how to set "sort oder" of the query?
> I tried to 'LiteralGqlQuery' to read data but DataStore IO Reader failed by
> follow error trace..
> {noformat}
> java.lang.IllegalArgumentException: Query cannot have any sort orders. at
> com.google.datastore.v1.client.QuerySplitterImpl.validateQuery(QuerySplitterImpl.java:128)
> ~[datastore-v1-proto-client-1.6.0.jar:na] at
> com.google.datastore.v1.client.QuerySplitterImpl.getSplits(QuerySplitterImpl.java:69)
> ~[datastore-v1-proto-client-1.6.0.jar:na] at
> org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.splitQuery(DatastoreV1.java:454)
> ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at
> org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read.access$100(DatastoreV1.java:264)
> ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at
> org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn.processElement(DatastoreV1.java:813)
> ~[beam-sdks-java-io-google-cloud-platform-2.8.0.jar:na] at
> org.apache.beam.sdk.io.gcp.datastore.DatastoreV1$Read$SplitQueryFn$DoFnInvoker.invokeProcessElement(Unknown
> Source) [na:na] at
> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:275)
> [beam-runners-direct-java-2.8.0.jar:na] at
> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:240)
> [beam-runners-direct-java-2.8.0.jar:na] at
> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:78)
> [beam-runners-direct-java-2.8.0.jar:na]
> at
> org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:207)
> [beam-runners-direct-java-2.8.0.jar:na] at
> org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:55)
> [beam-runners-direct-java-2.8.0.jar:na] at
> org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160)
> [beam-runners-direct-java-2.8.0.jar:na] at
> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124)
> [beam-runners-direct-java-2.8.0.jar:na] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_162] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_162] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_162] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_162] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
> {noformat}
> My code is here
>
> {code:java}
> PipelineOptions options = PipelineOptionsFactory.create();
> options.setRunner(DirectRunner.class); String gqlQuery =
> String.format("SELECT * FROM task " + "WHERE createdAt > DATETIME('%s') "+
> "AND createdAt < DATETIME('%s') ORDER BY createdAt ASC",
> "2018-11-03T00:00:00Z", "2018-11-03T23:59:59Z" ); log.info("GQL: " +
> gqlQuery); String projectId = "foo-bar"; String ns =
> "prod-01ce205a-22ff-4bab-b133-926bdc54c8b3";
> Pipeline p = Pipeline.create(options);
> PCollection<String> entities = p.apply( DatastoreIO.v1().read()
> .withProjectId(projectId).withLiteralGqlQuery(gqlQuery) .withQuery(query)
> .withNamespace(ns)) .apply("Debug", ParDo.of(new DoFn<Entity, String>(){
> @ProcessElement
> public void processElement(ProcessContext c) {
>
> log.info(c.element().getProperties().get("foo").getStringValue());
> }
> }));
> p.run().waitUntilFinish();
> {code}
> ```
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
