Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-22 Thread via GitHub
cloud-fan closed pull request #46040: [SPARK-47413][SQL] - add support to substr/left/right for collations URL: https://github.com/apache/spark/pull/46040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-22 Thread via GitHub
cloud-fan commented on PR #46040: URL: https://github.com/apache/spark/pull/46040#issuecomment-2068657667 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-21 Thread via GitHub
GideonPotok commented on PR #46040: URL: https://github.com/apache/spark/pull/46040#issuecomment-2068213773 @uros-db -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-19 Thread via GitHub
GideonPotok commented on PR #46040: URL: https://github.com/apache/spark/pull/46040#issuecomment-2065859992 @uros-db I have made the suggested changes. please re-review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-19 Thread via GitHub
GideonPotok commented on code in PR #46040: URL: https://github.com/apache/spark/pull/46040#discussion_r1571898438 ## sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala: ## @@ -425,6 +425,54 @@ class CollationStringExpressionsSuite }) }

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-18 Thread via GitHub
uros-db commented on code in PR #46040: URL: https://github.com/apache/spark/pull/46040#discussion_r1571028944 ## sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala: ## @@ -425,6 +425,54 @@ class CollationStringExpressionsSuite }) } +

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-18 Thread via GitHub
uros-db commented on code in PR #46040: URL: https://github.com/apache/spark/pull/46040#discussion_r1571030478 ## sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala: ## @@ -425,6 +425,54 @@ class CollationStringExpressionsSuite }) } +

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-18 Thread via GitHub
uros-db commented on code in PR #46040: URL: https://github.com/apache/spark/pull/46040#discussion_r1571028944 ## sql/core/src/test/scala/org/apache/spark/sql/CollationStringExpressionsSuite.scala: ## @@ -425,6 +425,54 @@ class CollationStringExpressionsSuite }) } +

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-14 Thread via GitHub
GideonPotok closed pull request #45738: [SPARK-47413][SQL] - add support to substr/left/right for collations URL: https://github.com/apache/spark/pull/45738 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-14 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2053959041 @ur. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
uros-db commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2048962946 @GideonPotok You are correct, this refactor should not greatly affect your current PR in particular - I expect you'll only need to refactor testing a bit (shouldn't be too much work)

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2048944931 PS: Do you think changes, such as these, which are only to implementations of `inputTypes` and `replacement`, which do not rely on calling UTFString or CollationFactory, will need to

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2048941620 @uros-db No problem at all. if I understand your refactor correctly, my changes will basically either stay in the same place or move to the new

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
uros-db commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2048923852 @GideonPotok nice work, thanks! Heads up though: we will soon be finishing some code refactoring related to collation-aware string expression support

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2048919774 @uros-db this is ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2047328959 @uros-db I got the file-writing tests to work locally when I simply `export SPARK_HOME=/Users/gideon/repos/spark` prior to running my maven tests. More importantly, All GHA

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2046941212 > I don't think `CSVLegacyTimeParserSuite` is related to you, but it would probably be a very good idea to setup Maven so that you can run/debug all tests locally in general

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
uros-db commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2046875237 I don't think `CSVLegacyTimeParserSuite` is related to you, but it would probably be a very good idea to setup Maven so that you can run/debug all tests locally in general -- This is

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-10 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2046822456 > did you try to run this suite locally and investigate any potential issues? @uros-db I would love to. I need to fix my setup though, first -- there is an issue I have been

Re: [PR] [SPARK-47413][SQL] - add support to substr/left/right for collations [spark]

2024-04-09 Thread via GitHub
GideonPotok commented on PR #45738: URL: https://github.com/apache/spark/pull/45738#issuecomment-2045605623 @uros-db this would be ready for review but spark sql unit tests repeatedly fail for CSVLegacyTimeParserSuite, which I did not modify. I have reran it a total of four times. Any idea