[
https://issues.apache.org/jira/browse/BEAM-14004?focusedWorklogId=774276&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-774276
]
ASF GitHub Bot logged work on BEAM-14004:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/May/22 22:57
Start Date: 24/May/22 22:57
Worklog Time Spent: 10m
Work Description: Miuler commented on code in PR #17511:
URL: https://github.com/apache/beam/pull/17511#discussion_r880629853
##########
sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbIO.java:
##########
@@ -489,8 +490,16 @@ public List<BoundedSource<Document>> split(
// maxChunkSize is the Mongo partition size in MB
LOG.debug("Splitting in chunk of {} MB", desiredBundleSizeBytes /
1024 / 1024);
splitVectorCommand.append("maxChunkSize", desiredBundleSizeBytes /
1024 / 1024);
- Document splitVectorCommandResult =
mongoDatabase.runCommand(splitVectorCommand);
- splitKeys = (List<Document>)
splitVectorCommandResult.get("splitKeys");
+ try {
Review Comment:
It is the official cosmos, try to use versions less than 2.6, to see if that
command does not exist in that old version, but those versions are no longer
supported by the mongo driver.
https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo/blob/master/src/main/java/de/flapdoodle/embed/mongo/distribution/Version.java
Issue Time Tracking
-------------------
Worklog Id: (was: 774276)
Time Spent: 10h 10m (was: 10h)
> [CosmosDB] com.mongodb.MongoCommandException: Command failed with error 115:
> Command is not supported
> -----------------------------------------------------------------------------------------------------
>
> Key: BEAM-14004
> URL: https://issues.apache.org/jira/browse/BEAM-14004
> Project: Beam
> Issue Type: Bug
> Components: io-java-mongodb
> Affects Versions: 2.36.0
> Reporter: Hector Miuler Malpica Gallegos
> Priority: P2
> Fix For: 2.36.0
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> You can't read cosmosdb collections using mongodb api because mongodb api
> doesn't support the *{{splitVector}}* command.
>
>
> {code:java}
> Sending command '{"splitVector": "test01.payment", "keyPattern": {"_id": 1},
> "force": false, "maxChunkSize": 1}' with request id 22 to database test01 on
> connection [connectionId{localValue:6, serverValue:792577514}] to server
> cdb-test.documents.azure.com:10255{code}
>
> {code:java}
> com.mongodb.MongoCommandException: Command failed with error 115: 'Command is
> not supported. ConnectionId fb598622-cc64-439a-affe-188c280c8e52
> ActivityId: fde0dba1-0000-0000-0000-000000000000' on server
> cdb-test.documents.azure.com:10255. The full response is{"_t":
> "OKMongoResponse", "ok": 0, "code": 115, "errmsg": "Command is not
> supported. ConnectionId fb598622-cc64-439a-affe-188c280c8e52 ActivityId:
> fde0dba1-0000-0000-0000-000000000000", "$err": "Command is not supported.
> ConnectionId fb598622-cc64-439a-affe-188c280c8e52 ActivityId:
> fde0dba1-0000-0000-0000-000000000000"} at
> com.mongodb.internal.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:175)
> at
> com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:303)
> at
> com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259)
> at
> com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
> at
> com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450)
> at
> com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
> at
> com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226)
> at
> com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
> at
> com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
> at
> com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220)
> at
> com.mongodb.operation.CommandOperationHelper$5.call(CommandOperationHelper.java:206)
> at
> com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:203)
> at
> com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:198)
> at
> com.mongodb.operation.CommandReadOperation.execute(CommandReadOperation.java:59)
> at
> com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:194)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.executeCommand(MongoDatabaseImpl.java:194)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:163)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:158)
> at
> com.mongodb.client.internal.MongoDatabaseImpl.runCommand(MongoDatabaseImpl.java:148)
> at
> org.apache.beam.sdk.io.mongodb.MongoDbIO$BoundedMongoDbSource.split(MongoDbIO.java:492)
> at
> org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$InputProvider.getInitialInputs(BoundedReadEvaluatorFactory.java:216)
> at
> org.apache.beam.runners.direct.ReadEvaluatorFactory$InputProvider.getInitialInputs(ReadEvaluatorFactory.java:88)
> at
> org.apache.beam.runners.direct.RootProviderRegistry.getInitialInputs(RootProviderRegistry.java:80)
> at
> org.apache.beam.runners.direct.ExecutorServiceParallelExecutor.start(ExecutorServiceParallelExecutor.java:160)
> at
> org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:213)
> at
> org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:309)
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
