andygrove merged PR #750:
URL: https://github.com/apache/datafusion-python/pull/750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@d
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1680924433
##
benchmarks/db-benchmark/join-datafusion.py:
##
@@ -74,7 +74,8 @@ def ans_shape(batches):
ctx = df.SessionContext()
print(ctx)
-# TODO we should be appl
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1680924793
##
python/datafusion/__init__.py:
##
@@ -15,206 +15,74 @@
# specific language governing permissions and limitations
# under the License.
-from abc import
Michael-J-Ward commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1680305302
##
benchmarks/db-benchmark/join-datafusion.py:
##
@@ -74,7 +74,8 @@ def ans_shape(batches):
ctx = df.SessionContext()
print(ctx)
-# TODO we should be
Michael-J-Ward commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1680305302
##
benchmarks/db-benchmark/join-datafusion.py:
##
@@ -74,7 +74,8 @@ def ans_shape(batches):
ctx = df.SessionContext()
print(ctx)
-# TODO we should be
Michael-J-Ward commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1680309889
##
python/datafusion/__init__.py:
##
@@ -15,206 +15,74 @@
# specific language governing permissions and limitations
# under the License.
-from abc im
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2232207709
Added a temporary work around for CI issues and added an issue to fix once
the upstream is resolved. https://github.com/apache/datafusion-python/issues/763
--
This is an auto
kylebarron commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2229872515
> I am currently testing mkdocs per @kylebarron recommendation.
Given that this project is currently using sphinx for documentation, it
would be a pretty big change to m
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2228513066
Updated all docstrings to match google format and added line length check
into pre-commit configuration. The only thing I have left on my TODO list is
updating the documentatio
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676818082
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676816902
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676814428
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676814381
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1676813966
##
python/datafusion/substrait.py:
##
@@ -15,9 +15,156 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675874679
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675874142
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675873313
##
python/datafusion/substrait.py:
##
@@ -15,9 +15,156 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675871415
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675870247
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675856084
##
python/datafusion/substrait.py:
##
@@ -15,9 +15,156 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1675852887
##
python/datafusion/udf.py:
##
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2224424124
Thanks @simicd, this is very helpful. I was assuming they'd use the
`__doc__` attribute, but seems like VS Code is internally using pylance which
is static code analyzer a
simicd commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223422816
Of course, here's what VS Code displays when hovering over it
Dataframe:
![image](https://github.com/apache/datafusion-python/assets/10134699/31e20c15-0612-497f-8ba6-f030
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223407581
@simicd if it's easy for you, do you mind checking what documentation is
shown for `datafusion.DataFrame`? I don't know too much about VS Code, I don't
use it myself, but
simicd commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223373598
First of all, thanks a lot for putting in all the work and drafting this PR
@timsaucer!!
I installed the package you mentioned @kylebarron to see how it behaves in
an IDE.
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223182914
I've been doing some research, and I think most of the usability
improvements I suggested above I think they should be better done in the
Rust/PyO3 code, not in Python.
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223145416
@austin362667 Thanks! I've been watching that with a lot of interest. The
original issue is over 2 years old and still making slow progress. I've dove
into some parts of it mys
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223133653
Proposal: To get a better feeling for the consensus of the group, can I get
a reaction to this comment:
thumbs up: recommend keep going with python wrappers as in this PR
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2223128481
@kylebarron I hadn't thought before about splitting up the type annotations
in one place and the documentation in another. That's very clever.
Also there was this comment
kylebarron commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2221797838
👋 I'm also building Python-Rust-Arrow libraries and thought it might be
useful to chime in.
Ultimately it's a personal preference, so I can't blame you if you all
choos
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2221529664
@datapythonista I love those suggestions. I'll get on it as soon as I can
get through some of the other comments.
--
This is an automated message from the Apache Git Service.
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220954392
Thanks for the comments, and sorry if my feedback is not helpful. Just one
last comment if you don't mind.
Since the final goal seems to be adoption, and making thin
austin362667 commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220885390
As per `.pyi` discussion, I think it's relevant and worth mentioning this
issue https://github.com/PyO3/pyo3/issues/2454 and also keep track the progress
of it.
--
This i
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220606862
@datapythonista I think your PR is great! It definitely cleans up a problem,
but it's not the same one as what I was trying to say.
I know there are many users who look a
Michael-J-Ward commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220558214
The wrappers also allow us to offer more pythonic interfaces when the `rust`
API is "clunky", at least by `python` standards.
An example of this is how @timsaucer cl
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220543813
Thanks for the detailed information @timsaucer, and for all the work with
this.
I'm not an expert on type hints but it's not immediately clear to me what
would ever
Michael-J-Ward commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671267473
##
src/common/data_type.rs:
##
@@ -764,7 +764,7 @@ pub enum SqlType {
#[allow(non_camel_case_types)]
#[allow(clippy::upper_case_acronyms)]
#[derive(De
timsaucer commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2220507454
It's a well made point @datapythonista and maintainability is definitely a
concern.
I would argue that going with a `.pyi` file approach is **worse** for
maintainability
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1672230398
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ impor
datapythonista commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2219816929
This looks great in terms of functionality, but I personally don't think
it's worth to have wrappers for every DataFusion functionality just for the
types and the docs. Af
Michael-J-Ward commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671263993
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __fut
Throne3d commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671173527
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
Throne3d commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671180646
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
Throne3d commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671173527
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
Throne3d commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671173527
##
python/datafusion/functions.py:
##
@@ -15,9 +15,1731 @@
# specific language governing permissions and limitations
# under the License.
+from __future__
slyons commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1671038077
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670896802
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ impor
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670892929
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ impor
emgeee commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670891866
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ import a
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670834793
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670834793
##
python/datafusion/context.py:
##
@@ -0,0 +1,1167 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670825912
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ impor
timsaucer commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670823051
##
python/datafusion/udf.py:
##
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670778616
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670779264
##
python/datafusion/expr.py:
##
@@ -15,9 +15,256 @@
# specific language governing permissions and limitations
# under the License.
+from __future__ impor
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670778616
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670771945
##
python/datafusion/record_batch.py:
##
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreeme
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670758586
##
python/datafusion/udf.py:
##
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See
max-muoto commented on code in PR #750:
URL: https://github.com/apache/datafusion-python/pull/750#discussion_r1670758586
##
python/datafusion/udf.py:
##
@@ -0,0 +1,62 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See
andygrove commented on PR #750:
URL:
https://github.com/apache/datafusion-python/pull/750#issuecomment-2217977268
@jdye64 @charlesbluca fyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
timsaucer opened a new pull request, #750:
URL: https://github.com/apache/datafusion-python/pull/750
# Which issue does this PR close?
Closes #749.
# Rationale for this change
As described the the [issue linked
above](https://github.com/apache/datafusion-python/issues/7
61 matches
Mail list logo
