milenkovicm commented on code in PR #1100: URL: https://github.com/apache/datafusion-ballista/pull/1100#discussion_r1832351156
########## python/examples/example.py: ########## @@ -0,0 +1,37 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from ballista import StandaloneBallista, RemoteBallista, Ballista Review Comment: why do we need ` StandaloneBallista` and `RemoteBallista,` if we go with `Ballista` builder ? ########## python/ballista/context.py: ########## @@ -0,0 +1,31 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from datafusion import SessionContext +import ballista + +from typing import List, Any + +class Ballista: + def __init__(self): + self.ctx = ballista.Ballista + Review Comment: would it be possible to expose `.conf(str, str)` option as well? ########## python/README.md: ########## @@ -29,7 +29,7 @@ part of the default Cargo workspace so that it doesn't cause overhead for mainta Creates a new context and connects to a Ballista scheduler process. ```python -from pyballista import SessionContext +from ballista import RemoteBallista, StandaloneBallista Review Comment: this readme does not reflect `Ballista` builder pattern ########## docs/source/user-guide/python.md: ########## @@ -103,14 +109,15 @@ The `explain` method can be used to show the logical and physical query plans fo The following example demonstrates creating arrays with PyArrow and then creating a Ballista DataFrame. ```python -import ballista +from ballista import Ballista import pyarrow # an alias +# TODO implement Functions f = ballista.functions # create a context -ctx = ballista.BallistaContext("localhost", 50050) +ctx = StandaloneBallista.build() Review Comment: should this be `Ballista.standalone()` ? ########## docs/source/user-guide/python.md: ########## @@ -28,9 +28,15 @@ popular file formats files, run it in a distributed environment, and obtain the The following code demonstrates how to create a Ballista context and connect to a scheduler. +If you are running a standalone cluster (runs locally), all you need to do is call the stand alone cluster method `standalone()` or your BallistaContext. If you are running a cluster in remote mode, you need to provide the URL `BallistaContext.remote("http://my-remote-ip:50050";)`. Review Comment: why do we have reference to `BallistaContext` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
