(arrow) branch main updated (23e1f4c2db -> 832880a6ed)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 23e1f4c2db GH-42025: [Java] Update Unit Tests for Algorithm Module (#42029) add 832880a6ed GH-40824: [Java] Adding Spotless to Dataset module (#42062) No new revisions were added by this update. Summary of changes: java/dataset/pom.xml | 3 + java/dataset/src/main/java/module-info.java| 30 +- .../arrow/dataset/file/DatasetFileWriter.java | 28 +- .../org/apache/arrow/dataset/file/FileFormat.java | 5 +- .../dataset/file/FileSystemDatasetFactory.java | 14 +- .../org/apache/arrow/dataset/file/JniWrapper.java | 22 +- .../dataset/jni/DirectReservationListener.java | 38 +- .../arrow/dataset/jni/JniExceptionDescriber.java | 12 +- .../org/apache/arrow/dataset/jni/JniLoader.java| 29 +- .../org/apache/arrow/dataset/jni/JniWrapper.java | 30 +- .../apache/arrow/dataset/jni/NativeContext.java| 13 +- .../apache/arrow/dataset/jni/NativeDataset.java| 18 +- .../arrow/dataset/jni/NativeDatasetFactory.java| 33 +- .../jni/NativeInstanceReleasedException.java | 5 +- .../apache/arrow/dataset/jni/NativeMemoryPool.java | 23 +- .../apache/arrow/dataset/jni/NativeScanTask.java | 12 +- .../apache/arrow/dataset/jni/NativeScanner.java| 24 +- .../arrow/dataset/jni/ReservationListener.java | 9 +- .../arrow/dataset/scanner/ArrowScannerReader.java | 10 +- .../apache/arrow/dataset/scanner/ScanOptions.java | 46 +- .../org/apache/arrow/dataset/scanner/ScanTask.java | 11 +- .../org/apache/arrow/dataset/scanner/Scanner.java | 9 +- .../org/apache/arrow/dataset/source/Dataset.java | 5 +- .../arrow/dataset/source/DatasetFactory.java | 10 +- .../dataset/substrait/AceroSubstraitConsumer.java | 61 +- .../apache/arrow/dataset/substrait/JniWrapper.java | 42 +- .../org/apache/arrow/dataset/OrcWriteSupport.java | 8 +- .../apache/arrow/dataset/ParquetWriteSupport.java | 41 +- .../org/apache/arrow/dataset/TestAllTypes.java | 140 +++-- .../java/org/apache/arrow/dataset/TestDataset.java | 34 +- .../arrow/dataset/TextBasedWriteSupport.java | 15 +- .../arrow/dataset/file/TestDatasetFileWriter.java | 76 +-- .../arrow/dataset/file/TestFileSystemDataset.java | 339 +++ .../dataset/file/TestFileSystemDatasetFactory.java | 32 +- .../arrow/dataset/jni/TestNativeDataset.java | 1 - .../arrow/dataset/jni/TestReservationListener.java | 81 +-- .../substrait/TestAceroSubstraitConsumer.java | 652 + 37 files changed, 1115 insertions(+), 846 deletions(-)
(arrow-adbc) branch dependabot/go_modules/go/adbc/google.golang.org/grpc-1.64.0 updated (4d639eefe -> 0992913bf)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/go_modules/go/adbc/google.golang.org/grpc-1.64.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from 4d639eefe --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... add 0992913bf Remove deprecated option No new revisions were added by this update. Summary of changes: docs/source/driver/flight_sql.rst | 9 + go/adbc/driver/flightsql/flightsql_database.go | 19 ++- go/adbc/driver/flightsql/flightsql_driver.go | 3 --- 3 files changed, 7 insertions(+), 24 deletions(-)
(arrow) branch main updated: GH-42025: [Java] Update Unit Tests for Algorithm Module (#42029)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 23e1f4c2db GH-42025: [Java] Update Unit Tests for Algorithm Module (#42029) 23e1f4c2db is described below commit 23e1f4c2db3501173d1e9b3a77bdc36b4c7b5bf4 Author: Hyunseok Seo AuthorDate: Wed Jun 12 14:19:33 2024 +0900 GH-42025: [Java] Update Unit Tests for Algorithm Module (#42029) ### Rationale for this change Update package from JUnit 4(`org.junit`) to JUnit 5(`org.junit.jupiter`). ### What changes are included in this PR? - [x] Replacing `org.junit` and `junit.framework.TestCase` with `org.junit.jupiter.api`. - [x] Updating `Assertions.assertXXX` to `assertXXX` using static imports. - [x] Updating annotations such as `@ Before`, `@ After`. - `@ Before` -> `@ BeforeEach` - `@ After` -> `@ AfterEach` - `@ Test` -> `@ Test` with `org.junit.jupiter` - [x] Updating `Parameterized` test - [x] Doing self review round 1 - [x] Resolving conflcit - [x] Doing self review rount 2 ### Are these changes tested? Yes, existing tests have passed. ### Are there any user-facing changes? No. * GitHub Issue: #42025 Authored-by: Hyunseok Seo Signed-off-by: David Li --- .../deduplicate/TestDeduplicationUtils.java| 12 +- .../deduplicate/TestVectorRunDeduplicator.java | 12 +- .../TestHashTableBasedDictionaryBuilder.java | 14 +- .../dictionary/TestHashTableDictionaryEncoder.java | 12 +- .../dictionary/TestLinearDictionaryEncoder.java| 19 ++- .../dictionary/TestSearchDictionaryEncoder.java| 19 ++- .../TestSearchTreeBasedDictionaryBuilder.java | 14 +- .../arrow/algorithm/misc/TestPartialSumUtils.java | 12 +- .../arrow/algorithm/rank/TestVectorRank.java | 12 +- .../algorithm/search/TestParallelSearcher.java | 54 +++ .../algorithm/search/TestVectorRangeSearcher.java | 51 +++--- .../arrow/algorithm/search/TestVectorSearcher.java | 12 +- .../sort/TestCompositeVectorComparator.java| 10 +- .../sort/TestDefaultVectorComparator.java | 17 +- .../sort/TestFixedWidthInPlaceVectorSorter.java| 37 +++-- .../sort/TestFixedWidthOutOfPlaceVectorSorter.java | 178 +++-- .../algorithm/sort/TestFixedWidthSorting.java | 168 +-- .../sort/TestGeneralOutOfPlaceVectorSorter.java| 12 +- .../arrow/algorithm/sort/TestIndexSorter.java | 14 +- .../arrow/algorithm/sort/TestInsertionSorter.java | 12 +- .../arrow/algorithm/sort/TestOffHeapIntStack.java | 14 +- .../algorithm/sort/TestOutOfPlaceVectorSorter.java | 24 +-- .../algorithm/sort/TestStableVectorComparator.java | 15 +- .../TestVariableWidthOutOfPlaceVectorSorter.java | 35 ++-- .../algorithm/sort/TestVariableWidthSorting.java | 67 +++- 25 files changed, 396 insertions(+), 450 deletions(-) diff --git a/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestDeduplicationUtils.java b/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestDeduplicationUtils.java index 537189013a..49a719bafa 100644 --- a/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestDeduplicationUtils.java +++ b/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestDeduplicationUtils.java @@ -16,8 +16,8 @@ */ package org.apache.arrow.algorithm.deduplicate; -import static org.junit.Assert.assertEquals; import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; import java.nio.charset.StandardCharsets; import org.apache.arrow.memory.ArrowBuf; @@ -27,9 +27,9 @@ import org.apache.arrow.vector.BitVectorHelper; import org.apache.arrow.vector.IntVector; import org.apache.arrow.vector.VarCharVector; import org.apache.arrow.vector.util.DataSizeRoundingUtil; -import org.junit.After; -import org.junit.Before; -import org.junit.Test; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; /** Test cases for {@link DeduplicationUtils}. */ public class TestDeduplicationUtils { @@ -40,12 +40,12 @@ public class TestDeduplicationUtils { private BufferAllocator allocator; - @Before + @BeforeEach public void prepare() { allocator = new RootAllocator(1024 * 1024); } - @After + @AfterEach public void shutdown() { allocator.close(); } diff --git a/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestVectorRunDeduplicator.java b/java/algorithm/src/test/java/org/apache/arrow/algorithm/deduplicate/TestVectorRunDeduplicator.java index 820cadccae..7fd816ee9f 100644 --- a/java/algorithm/src/test/java/org/apa
(arrow) branch main updated (c7bfd2e996 -> ee62d97033)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from c7bfd2e996 GH-40832: [Java] Adding Spotless to Vector module (#42061) add ee62d97033 GH-40829: [Java] Adding Spotless to Memory modules (#42056) No new revisions were added by this update. Summary of changes: java/dev/checkstyle/checkstyle-spotless.xml| 3 + .../memory-core/src/main/java/module-info.java | 1 + .../java/org/apache/arrow/memory/Accountant.java | 118 +++-- .../apache/arrow/memory/AllocationListener.java| 28 +- .../org/apache/arrow/memory/AllocationManager.java | 105 +++-- .../org/apache/arrow/memory/AllocationOutcome.java | 30 +- .../arrow/memory/AllocationOutcomeDetails.java | 58 ++- .../apache/arrow/memory/AllocationReservation.java | 26 +- .../arrow/memory/AllocatorClosedException.java | 6 +- .../java/org/apache/arrow/memory/ArrowBuf.java | 498 +++-- .../org/apache/arrow/memory/BaseAllocator.java | 299 +++-- .../org/apache/arrow/memory/BoundsChecking.java| 25 +- .../org/apache/arrow/memory/BufferAllocator.java | 96 ++-- .../java/org/apache/arrow/memory/BufferLedger.java | 221 + .../org/apache/arrow/memory/BufferManager.java | 13 +- .../org/apache/arrow/memory/CheckAllocator.java| 26 +- .../org/apache/arrow/memory/ChildAllocator.java| 21 +- .../memory/DefaultAllocationManagerOption.java | 58 +-- .../org/apache/arrow/memory/ForeignAllocation.java | 16 +- .../arrow/memory/ForeignAllocationManager.java | 8 +- .../arrow/memory/LowCostIdentityHashMap.java | 68 ++- .../apache/arrow/memory/OutOfMemoryException.java | 17 +- .../apache/arrow/memory/OwnershipTransferNOOP.java | 5 +- .../arrow/memory/OwnershipTransferResult.java | 5 +- .../org/apache/arrow/memory/ReferenceManager.java | 169 +++ .../org/apache/arrow/memory/ReusableBuffer.java| 17 +- .../org/apache/arrow/memory/RootAllocator.java | 33 +- .../apache/arrow/memory/ValueWithKeyIncluded.java | 4 +- .../java/org/apache/arrow/memory/package-info.java | 6 +- .../memory/rounding/DefaultRoundingPolicy.java | 34 +- .../arrow/memory/rounding/RoundingPolicy.java | 1 - .../memory/rounding/SegmentRoundingPolicy.java | 28 +- .../apache/arrow/memory/util/ArrowBufPointer.java | 46 +- .../apache/arrow/memory/util/AssertionUtil.java| 7 +- .../arrow/memory/util/AutoCloseableLock.java | 6 +- .../arrow/memory/util/ByteFunctionHelpers.java | 85 ++-- .../org/apache/arrow/memory/util/CommonUtil.java | 9 +- .../java/org/apache/arrow/memory/util/Float16.java | 103 ++--- .../apache/arrow/memory/util/HistoricalLog.java| 78 ++-- .../apache/arrow/memory/util/LargeMemoryUtil.java | 13 +- .../org/apache/arrow/memory/util/MemoryUtil.java | 131 +++--- .../org/apache/arrow/memory/util/StackTrace.java | 15 +- .../arrow/memory/util/hash/ArrowBufHasher.java | 12 +- .../arrow/memory/util/hash/MurmurHasher.java | 32 +- .../arrow/memory/util/hash/SimpleHasher.java | 35 +- .../java/org/apache/arrow/util/AutoCloseables.java | 76 ++-- .../java/org/apache/arrow/util/Collections2.java | 29 +- .../java/org/apache/arrow/util/Preconditions.java | 264 --- .../org/apache/arrow/util/VisibleForTesting.java | 9 +- .../arrow/memory/CountingAllocationListener.java | 18 +- .../memory/DefaultAllocationManagerFactory.java| 10 +- .../org/apache/arrow/memory/TestAccountant.java| 29 +- .../apache/arrow/memory/TestAllocationManager.java | 5 +- .../java/org/apache/arrow/memory/TestArrowBuf.java | 33 +- .../org/apache/arrow/memory/TestBaseAllocator.java | 347 +++--- .../apache/arrow/memory/TestBoundaryChecking.java | 21 +- .../apache/arrow/memory/TestForeignAllocation.java | 27 +- .../arrow/memory/TestLowCostIdentityHashMap.java | 13 +- .../java/org/apache/arrow/memory/TestOpens.java| 15 +- .../arrow/memory/util/TestArrowBufPointer.java | 13 +- .../arrow/memory/util/TestByteFunctionHelpers.java | 74 ++- .../arrow/memory/util/TestLargeMemoryUtil.java | 215 - .../arrow/memory/util/hash/TestArrowBufHasher.java | 55 ++- .../org/apache/arrow/util/TestCollections2.java| 8 +- .../java/org/apache/arrow/util/TestStackTrace.java | 12 +- .../java/io/netty/buffer/ExpandableByteBuf.java| 5 +- .../src/main/java/io/netty/buffer/LargeBuffer.java | 1 - .../io/netty/buffer/MutableWrappedByteBuf.java | 28 +- .../main/java/io/netty/buffer/NettyArrowBuf.java | 91 ++-- .../io/netty/buffer/PooledByteBufAllocatorL.java | 47 +- .../io/netty/buffer/UnsafeDirectLittleEndian.java | 9 +- .../arrow/memory/patch/ArrowByteBufAllocator.java | 16 +- .../netty/buffer/TestUnsafeDirectLittleEndian.java | 10 +- .../netty/DefaultAllocationManagerFactory.java
(arrow) branch dependabot/maven/java/maven.version-3.9.7 deleted (was 8990d582b0)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/maven.version-3.9.7 in repository https://gitbox.apache.org/repos/asf/arrow.git was 8990d582b0 MINOR: [Java] Bump maven.version from 3.8.7 to 3.9.7 in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated: GH-40823: [Java] Adding Spotless to Compression module (#42060)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 1a48366924 GH-40823: [Java] Adding Spotless to Compression module (#42060) 1a48366924 is described below commit 1a48366924a1ca77df62d7406b44b8eb76bfc54b Author: Vibhatha Lakmal Abeykoon AuthorDate: Tue Jun 11 12:22:04 2024 +0530 GH-40823: [Java] Adding Spotless to Compression module (#42060) ### Rationale for this change Adding code style and formatting options for Compression module. ### What changes are included in this PR? Code formatting spotless plugin has been added. ### Are these changes tested? Yes, but doesn't involve test cases, the plugin itself corrects. * GitHub Issue: #40823 Authored-by: Vibhatha Abeykoon Signed-off-by: David Li --- java/compression/pom.xml | 5 + .../compression/CommonsCompressionFactory.java | 3 +- .../arrow/compression/Lz4CompressionCodec.java | 31 ++-- .../arrow/compression/ZstdCompressionCodec.java| 36 +++-- .../TestArrowReaderWriterWithCompression.java | 133 +-- .../arrow/compression/TestCompressionCodec.java| 179 + 6 files changed, 234 insertions(+), 153 deletions(-) diff --git a/java/compression/pom.xml b/java/compression/pom.xml index 561877bd5c..3b4449c33f 100644 --- a/java/compression/pom.xml +++ b/java/compression/pom.xml @@ -28,6 +28,11 @@ under the License. Arrow Compression (Experimental/Contrib) A library for working with the compression/decompression of Arrow data. + + dev/checkstyle/checkstyle-spotless.xml +none + + org.apache.arrow diff --git a/java/compression/src/main/java/org/apache/arrow/compression/CommonsCompressionFactory.java b/java/compression/src/main/java/org/apache/arrow/compression/CommonsCompressionFactory.java index 45d8c7d443..f15c139df1 100644 --- a/java/compression/src/main/java/org/apache/arrow/compression/CommonsCompressionFactory.java +++ b/java/compression/src/main/java/org/apache/arrow/compression/CommonsCompressionFactory.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.arrow.compression; import org.apache.arrow.vector.compression.CompressionCodec; @@ -23,7 +22,7 @@ import org.apache.arrow.vector.compression.CompressionUtil; /** * Default implementation of factory supported LZ4 and ZSTD compression. * - * // TODO(ARROW-12115): Rename this class. + * // TODO(ARROW-12115): Rename this class. */ public class CommonsCompressionFactory implements CompressionCodec.Factory { diff --git a/java/compression/src/main/java/org/apache/arrow/compression/Lz4CompressionCodec.java b/java/compression/src/main/java/org/apache/arrow/compression/Lz4CompressionCodec.java index e8b780638e..edd52604bc 100644 --- a/java/compression/src/main/java/org/apache/arrow/compression/Lz4CompressionCodec.java +++ b/java/compression/src/main/java/org/apache/arrow/compression/Lz4CompressionCodec.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.arrow.compression; import java.io.ByteArrayInputStream; @@ -22,7 +21,6 @@ import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; - import org.apache.arrow.memory.ArrowBuf; import org.apache.arrow.memory.BufferAllocator; import org.apache.arrow.util.Preconditions; @@ -32,21 +30,21 @@ import org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorInputStrea import org.apache.commons.compress.compressors.lz4.FramedLZ4CompressorOutputStream; import org.apache.commons.compress.utils.IOUtils; -/** - * Compression codec for the LZ4 algorithm. - */ +/** Compression codec for the LZ4 algorithm. */ public class Lz4CompressionCodec extends AbstractCompressionCodec { @Override protected ArrowBuf doCompress(BufferAllocator allocator, ArrowBuf uncompressedBuffer) { -Preconditions.checkArgument(uncompressedBuffer.writerIndex() <= Integer.MAX_VALUE, -"The uncompressed buffer size exceeds the integer limit %s.", Integer.MAX_VALUE); +Preconditions.checkArgument( +uncompressedBuffer.writerIndex() <= Integer.MAX_VALUE, +"The uncompressed buffer size exceeds the integer limit %s.", +Integer.MAX_VALUE); byte[] inBytes = new byte[(int) uncompressedBuffer.writerIndex()]; -uncompressedBuffer.getBytes(/*index=*/0, inBytes); +uncompressedBuffer.getBytes(/*index=*/ 0, inBytes); ByteArrayOutputStream baos = new ByteArrayOutputStream(); try (InputStream in = new By
(arrow) branch dependabot/maven/java/org.apache.orc-orc-core-2.0.1 deleted (was bbee978225)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.apache.orc-orc-core-2.0.1 in repository https://gitbox.apache.org/repos/asf/arrow.git was bbee978225 MINOR: [Java] Bump org.apache.orc:orc-core from 1.9.2 to 2.0.1 in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated (12e32f5347 -> ee6fcf3c46)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 12e32f5347 GH-40830: [Java] Adding Spotless to Performance module (#42057) add ee6fcf3c46 GH-40827: [Java] Adding Spotless to Gandiva module (#42055) No new revisions were added by this update. Summary of changes: java/gandiva/pom.xml | 2 + java/gandiva/src/main/java/module-info.java| 2 +- .../gandiva/evaluator/ConfigurationBuilder.java| 16 +- .../arrow/gandiva/evaluator/DecimalTypeUtil.java | 42 +- .../gandiva/evaluator/ExpressionRegistry.java | 45 +- .../evaluator/ExpressionRegistryJniHelper.java | 6 +- .../org/apache/arrow/gandiva/evaluator/Filter.java | 70 +- .../arrow/gandiva/evaluator/FunctionSignature.java | 30 +- .../apache/arrow/gandiva/evaluator/JniLoader.java | 55 +- .../apache/arrow/gandiva/evaluator/JniWrapper.java | 121 +-- .../apache/arrow/gandiva/evaluator/Projector.java | 277 +++--- .../arrow/gandiva/evaluator/SelectionVector.java | 15 +- .../gandiva/evaluator/SelectionVectorInt16.java| 5 +- .../gandiva/evaluator/SelectionVectorInt32.java| 5 +- .../arrow/gandiva/evaluator/VectorExpander.java| 16 +- .../exceptions/EvaluatorClosedException.java | 1 - .../arrow/gandiva/exceptions/GandivaException.java | 1 - .../exceptions/UnsupportedTypeException.java | 5 +- .../apache/arrow/gandiva/expression/AndNode.java | 6 +- .../arrow/gandiva/expression/ArrowTypeHelper.java | 463 +- .../arrow/gandiva/expression/BinaryNode.java | 17 +- .../arrow/gandiva/expression/BooleanNode.java | 5 +- .../apache/arrow/gandiva/expression/Condition.java | 6 +- .../arrow/gandiva/expression/DecimalNode.java | 5 +- .../arrow/gandiva/expression/DoubleNode.java | 5 +- .../arrow/gandiva/expression/ExpressionTree.java | 6 +- .../apache/arrow/gandiva/expression/FieldNode.java | 5 +- .../apache/arrow/gandiva/expression/FloatNode.java | 5 +- .../arrow/gandiva/expression/FunctionNode.java | 6 +- .../apache/arrow/gandiva/expression/IfNode.java| 5 +- .../apache/arrow/gandiva/expression/InNode.java| 129 +-- .../apache/arrow/gandiva/expression/IntNode.java | 5 +- .../apache/arrow/gandiva/expression/LongNode.java | 5 +- .../apache/arrow/gandiva/expression/NullNode.java | 1 - .../apache/arrow/gandiva/expression/OrNode.java| 6 +- .../arrow/gandiva/expression/StringNode.java | 20 +- .../arrow/gandiva/expression/TreeBuilder.java | 88 +- .../apache/arrow/gandiva/expression/TreeNode.java | 5 +- .../arrow/gandiva/evaluator/BaseEvaluatorTest.java | 51 +- .../gandiva/evaluator/DecimalTypeUtilTest.java | 44 +- .../gandiva/evaluator/ExpressionRegistryTest.java | 5 +- .../arrow/gandiva/evaluator/FilterProjectTest.java | 28 +- .../apache/arrow/gandiva/evaluator/FilterTest.java | 49 +- .../gandiva/evaluator/MicroBenchmarkTest.java | 54 +- .../gandiva/evaluator/ProjectorDecimalTest.java| 652 +++--- .../arrow/gandiva/evaluator/ProjectorTest.java | 932 ++--- .../arrow/gandiva/evaluator/TestJniLoader.java | 42 +- .../gandiva/expression/ArrowTypeHelperTest.java| 2 - .../arrow/gandiva/expression/TreeBuilderTest.java | 2 - 49 files changed, 1754 insertions(+), 1614 deletions(-)
(arrow) branch main updated: GH-40830: [Java] Adding Spotless to Performance module (#42057)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 12e32f5347 GH-40830: [Java] Adding Spotless to Performance module (#42057) 12e32f5347 is described below commit 12e32f53470098e14996730fa7fe9165159419d6 Author: Laurent Goujon AuthorDate: Mon Jun 10 17:40:54 2024 -0700 GH-40830: [Java] Adding Spotless to Performance module (#42057) ### Rationale for this change Applying Java code style and formatting options to Performance module. ### What changes are included in this PR? Java code formatting via spotless plugin has been enabled. ### Are these changes tested? Yes, but doesn't involve test cases, the plugin itself corrects. ### Are there any user-facing changes? No * GitHub Issue: #40830 Authored-by: Laurent Goujon Signed-off-by: David Li --- java/performance/pom.xml | 2 + .../arrow/adapter/AvroAdapterBenchmarks.java | 47 + .../arrow/adapter/jdbc/JdbcAdapterBenchmarks.java | 45 - .../search/ParallelSearcherBenchmarks.java | 13 ++--- .../apache/arrow/memory/AllocatorBenchmarks.java | 24 +++-- .../apache/arrow/memory/ArrowBufBenchmarks.java| 20 ++-- .../memory/util/ArrowBufPointerBenchmarks.java | 19 ++- .../memory/util/ByteFunctionHelpersBenchmarks.java | 38 -- .../arrow/vector/BaseValueVectorBenchmarks.java| 28 -- .../arrow/vector/BitVectorHelperBenchmarks.java| 59 -- .../arrow/vector/DecimalVectorBenchmarks.java | 33 .../org/apache/arrow/vector/Float8Benchmarks.java | 31 .../arrow/vector/FloatingPointBenchmarks.java | 20 +++- .../org/apache/arrow/vector/IntBenchmarks.java | 14 ++--- .../org/apache/arrow/vector/VarCharBenchmarks.java | 22 +++- .../vector/VariableWidthVectorBenchmarks.java | 21 +++- .../apache/arrow/vector/VectorLoaderBenchmark.java | 24 +++-- .../arrow/vector/VectorUnloaderBenchmark.java | 19 +++ .../dictionary/DictionaryEncoderBenchmarks.java| 29 --- .../arrow/vector/ipc/WriteChannelBenchmark.java| 16 ++ .../ipc/message/ArrowRecordBatchBenchmarks.java| 22 +++- .../arrow/vector/util/TransferPairBenchmarks.java | 22 +++- 22 files changed, 205 insertions(+), 363 deletions(-) diff --git a/java/performance/pom.xml b/java/performance/pom.xml index 83b0a88da0..1db900c81e 100644 --- a/java/performance/pom.xml +++ b/java/performance/pom.xml @@ -30,6 +30,8 @@ under the License. JMH Performance benchmarks for other Arrow libraries. + dev/checkstyle/checkstyle-spotless.xml +none 1.37 benchmarks true diff --git a/java/performance/src/main/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java b/java/performance/src/main/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java index c07aeffafb..6032a79ece 100644 --- a/java/performance/src/main/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java +++ b/java/performance/src/main/java/org/apache/arrow/adapter/AvroAdapterBenchmarks.java @@ -14,13 +14,11 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.arrow.adapter; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.util.concurrent.TimeUnit; - import org.apache.arrow.adapter.avro.AvroToArrow; import org.apache.arrow.adapter.avro.AvroToArrowConfig; import org.apache.arrow.adapter.avro.AvroToArrowConfigBuilder; @@ -52,9 +50,7 @@ import org.openjdk.jmh.runner.RunnerException; import org.openjdk.jmh.runner.options.Options; import org.openjdk.jmh.runner.options.OptionsBuilder; -/** - * Benchmarks for avro adapter. - */ +/** Benchmarks for avro adapter. */ @State(Scope.Benchmark) public class AvroAdapterBenchmarks { @@ -65,21 +61,25 @@ public class AvroAdapterBenchmarks { private Schema schema; private BinaryDecoder decoder; - /** - * Setup benchmarks. - */ + /** Setup benchmarks. */ @Setup public void prepare() throws Exception { BufferAllocator allocator = new RootAllocator(Integer.MAX_VALUE); config = new AvroToArrowConfigBuilder(allocator).build(); -String schemaStr = "{\n" + " \"namespace\": \"org.apache.arrow.avro\",\n" + - " \"type\": \"record\",\n" + " \"name\": \"testBenchmark\",\n" + " \"fields\": [\n" + - "{\"name\": \"f0\", \"type\": \"string\"},\n" + - "{\"name\": \"f1\", \"type\":
(arrow) branch main updated (8710bb2601 -> 2f4354d8c7)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 8710bb2601 MINOR: [Java] Bump org.codehaus.mojo:build-helper-maven-plugin from 1.9.1 to 3.6.0 in /java (#42079) add 2f4354d8c7 GH-40831: [Java] Adding Spotless to Tools module (#42058) No new revisions were added by this update. Summary of changes: java/tools/pom.xml | 5 + .../java/org/apache/arrow/tools/EchoServer.java| 26 +--- .../java/org/apache/arrow/tools/FileRoundtrip.java | 18 +-- .../java/org/apache/arrow/tools/FileToStream.java | 20 +-- .../java/org/apache/arrow/tools/Integration.java | 67 + .../java/org/apache/arrow/tools/StreamToFile.java | 18 +-- .../apache/arrow/tools/ArrowFileTestFixtures.java | 26 ++-- .../org/apache/arrow/tools/EchoServerTest.java | 104 +++--- .../org/apache/arrow/tools/TestFileRoundtrip.java | 6 +- .../org/apache/arrow/tools/TestIntegration.java| 155 ++--- 10 files changed, 233 insertions(+), 212 deletions(-)
(arrow-adbc) branch dependabot/go_modules/go/adbc/golang.org/x/tools-0.22.0 deleted (was 166c70240)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/go_modules/go/adbc/golang.org/x/tools-0.22.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 166c70240 chore(go/adbc): bump golang.org/x/tools in /go/adbc The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated: chore(go/adbc): bump golang.org/x/tools from 0.21.0 to 0.22.0 in /go/adbc (#1914)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new acef3c826 chore(go/adbc): bump golang.org/x/tools from 0.21.0 to 0.22.0 in /go/adbc (#1914) acef3c826 is described below commit acef3c826599c5d4c18849bfe2164d8d0ce53fbc Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon Jun 10 19:17:06 2024 -0400 chore(go/adbc): bump golang.org/x/tools from 0.21.0 to 0.22.0 in /go/adbc (#1914) Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.21.0 to 0.22.0. Commits https://github.com/golang/tools/commit/bc6931db37c33e064504346d9259b3b6d20e13f6;>bc6931d go.mod: update golang.org/x dependencies https://github.com/golang/tools/commit/1767b144a15d30b79c7744524828fe10bf07a862;>1767b14 go/ssa: remove code with no effect https://github.com/golang/tools/commit/208808308b705255ee693cdc55afe8b4ad37c425;>2088083 internal/gocommand: add more debug info for hanging go commands https://github.com/golang/tools/commit/f1a3b1281e9b5e25fc2f46e88e73023dbb4bbcf0;>f1a3b12 internal/imports: FixImports should be cancellable https://github.com/golang/tools/commit/4478db00aae5c5e487165f6e768681be5d63bac0;>4478db0 go/analysis/passes/copylock: suppress error in ill-typed code https://github.com/golang/tools/commit/018d3b2768ced9f36e90baeb2a542c813dfc28ef;>018d3b2 gopls: warn about Go 1.19 and Go 1.20 https://github.com/golang/tools/commit/58cc8a4458597b256f404b4a4a84edc71cd0d00b;>58cc8a4 gopls/internal/filecache: suppress gc in tests https://github.com/golang/tools/commit/b6235391adb3b7f8bcfc4df81055e8f023de2688;>b623539 gopls/internal/cache: suppress internal import check on Bazel https://github.com/golang/tools/commit/1e9d12dd1f25735a6fcefd3665be4684ba23fc58;>1e9d12d go/packages: pass -overlay to all 'go list' invocations https://github.com/golang/tools/commit/3c293ad67a98a86d273bf69c6a742f04a6a367a3;>3c293ad internal/cache: invalidate broken imports when package files change Additional commits viewable in https://github.com/golang/tools/compare/v0.21.0...v0.22.0;>compare view [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/tools=go_modules=0.21.0=0.22.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go/adbc/go.mod | 14 - go/adbc/go.sum | 94 ++ 2 files changed, 29 insertions(+), 79 deletions(-) diff --git a/go/adbc/go.mod b/go/adbc/go.mod index 302617a8e..5f479d79b 100644 --- a/go/adbc/go.mod +++ b/go/adbc/go.mod @@ -29,7 +29,7 @@ require ( github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a golang.org/x/exp v0.0.0-20240529
(arrow) branch main updated: GH-41929: [Java] pom.xml license formatting (#42049)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 0ff17bff3b GH-41929: [Java] pom.xml license formatting (#42049) 0ff17bff3b is described below commit 0ff17bff3ba9f4eb666e68fda4f5e04e4f009ef3 Author: Laurent Goujon AuthorDate: Mon Jun 10 05:37:01 2024 -0700 GH-41929: [Java] pom.xml license formatting (#42049) ### What changes are included in this PR? Reformat the XML license to match style defined at https://www.apache.org/legal/src-headers.html Include license formatting to all Maven `pom.xml` files via spotless plugin. ### Are these changes tested? Via CI. ### Are there any user-facing changes? No * GitHub Issue: #41929 Authored-by: Laurent Goujon Signed-off-by: David Li --- java/adapter/avro/pom.xml | 28 --- java/adapter/jdbc/pom.xml | 28 --- java/adapter/orc/pom.xml | 28 --- java/algorithm/pom.xml | 28 --- java/bom/pom.xml | 32 +++--- java/c/pom.xml | 28 --- java/compression/pom.xml | 28 --- java/dataset/pom.xml | 28 --- java/dev/license/asf-xml.license | 28 --- java/flight/flight-core/pom.xml| 28 --- java/flight/flight-integration-tests/pom.xml | 28 --- java/flight/flight-sql-jdbc-core/pom.xml | 28 --- java/flight/flight-sql-jdbc-driver/pom.xml | 28 --- java/flight/flight-sql/pom.xml | 28 --- java/flight/pom.xml| 28 --- java/format/pom.xml| 28 --- java/gandiva/pom.xml | 28 --- .../module-info-compiler-maven-plugin/pom.xml | 28 --- java/maven/pom.xml | 32 +++--- java/memory/memory-core/pom.xml| 28 --- java/memory/memory-netty-buffer-patch/pom.xml | 28 --- java/memory/memory-netty/pom.xml | 28 --- java/memory/memory-unsafe/pom.xml | 28 --- java/memory/pom.xml| 28 --- java/performance/pom.xml | 28 --- java/pom.xml | 32 +++--- java/tools/pom.xml | 28 --- java/vector/pom.xml| 28 --- 28 files changed, 516 insertions(+), 280 deletions(-) diff --git a/java/adapter/avro/pom.xml b/java/adapter/avro/pom.xml index 0af1641aa1..88aea71862 100644 --- a/java/adapter/avro/pom.xml +++ b/java/adapter/avro/pom.xml @@ -1,14 +1,22 @@ - + http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 diff --git a/java/adapter/jdbc/pom.xml b/java/adapter/jdbc/pom.xml index b444eff562..fc0f7e0927 100644 --- a/java/adapter/jdbc/pom.xml +++ b/java/adapter/jdbc/pom.xml @@ -1,14 +1,22 @@ - + http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 diff --git a/java/adapter/orc/pom.xml b/java/adapter/orc/pom.xml index 7df08e1a98..6c79ba8da4 100644 --- a/java/adapter/orc/pom.xml +++ b/java/adapter/orc/pom.xml @@ -1,14 +1,22 @@ - + http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 diff --git a/java/algorithm/pom.xml b/java/algorithm/pom.xml index 5984cce766..57e8fdc81b 100644 --- a/java/algorithm/pom.xml +++ b/java/algorithm/pom.xml @@ -1,14 +1,22 @@ - + http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> 4.0.0 diff --git a/java/bom/pom.xml b/java/bom/pom.xml index 77aed2d0f6..5fafbf38c7 100644 --- a/java/bom/pom.xml +++ b/java/bom/pom.xml @@ -1,14 +1,22 @@ - + http://maven.apac
(arrow) branch main updated: GH-40828: [Java] Format arrow-maven-plugins modules (#42054)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 9ea7f6f0c6 GH-40828: [Java] Format arrow-maven-plugins modules (#42054) 9ea7f6f0c6 is described below commit 9ea7f6f0c66af537ca6511de2df2a77078000158 Author: Laurent Goujon AuthorDate: Mon Jun 10 05:34:43 2024 -0700 GH-40828: [Java] Format arrow-maven-plugins modules (#42054) ### Rationale for this change Adding code style and formatting options for `apache-maven-plugin` module and submodules ### What changes are included in this PR? Java code formatting spotless plugin has been added. ### Are these changes tested? Yes, but doesn't involve test cases, the plugin itself corrects. ### Are there any user-facing changes? No * GitHub Issue: #40828 Authored-by: Laurent Goujon Signed-off-by: David Li --- .../plugins/BaseModuleInfoCompilerPlugin.java | 26 +- .../maven/plugins/ModuleInfoCompilerPlugin.java| 10 - .../plugins/ModuleInfoTestCompilerPlugin.java | 6 + java/maven/pom.xml | 12 +- 4 files changed, 27 insertions(+), 27 deletions(-) diff --git a/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/BaseModuleInfoCompilerPlugin.java b/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/BaseModuleInfoCompilerPlugin.java index 37cbf5d7e7..4fc8fc46e6 100644 --- a/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/BaseModuleInfoCompilerPlugin.java +++ b/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/BaseModuleInfoCompilerPlugin.java @@ -14,7 +14,6 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - package org.apache.arrow.maven.plugins; import java.io.File; @@ -28,14 +27,11 @@ import java.nio.file.Path; import java.nio.file.Paths; import java.util.List; import java.util.Optional; - import org.apache.maven.plugin.AbstractMojo; import org.apache.maven.plugin.MojoExecutionException; import org.glavo.mic.ModuleInfoCompiler; -/** - * Compiles the first module-info.java file in the project purely syntactically. - */ +/** Compiles the first module-info.java file in the project purely syntactically. */ public abstract class BaseModuleInfoCompilerPlugin extends AbstractMojo { protected abstract List getSourceRoots(); @@ -60,9 +56,10 @@ public abstract class BaseModuleInfoCompilerPlugin extends AbstractMojo { // Invoke the compiler, ModuleInfoCompiler compiler = new ModuleInfoCompiler(); - try (Reader reader = new InputStreamReader(Files.newInputStream(moduleInfoFile.get().toPath()), - StandardCharsets.UTF_8); - OutputStream output = Files.newOutputStream(targetPath)) { + try (Reader reader = + new InputStreamReader( + Files.newInputStream(moduleInfoFile.get().toPath()), StandardCharsets.UTF_8); + OutputStream output = Files.newOutputStream(targetPath)) { compiler.compile(reader, output); getLog().info("Successfully wrote module-info.class file."); } catch (IOException ex) { @@ -73,18 +70,17 @@ public abstract class BaseModuleInfoCompilerPlugin extends AbstractMojo { } } - /** - * Finds the first module-info.java file in the set of source directories. - */ + /** Finds the first module-info.java file in the set of source directories. */ private Optional findFirstModuleInfo(List sourceDirectories) { if (sourceDirectories == null) { return Optional.empty(); } -return sourceDirectories.stream().map(Paths::get) -.map(sourcePath -> -sourcePath.toFile().listFiles(file -> -file.getName().equals("module-info.java"))) +return sourceDirectories.stream() +.map(Paths::get) +.map( +sourcePath -> +sourcePath.toFile().listFiles(file -> file.getName().equals("module-info.java"))) .filter(matchingFiles -> matchingFiles != null && matchingFiles.length != 0) .map(matchingFiles -> matchingFiles[0]) .findAny(); diff --git a/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/ModuleInfoCompilerPlugin.java b/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/ModuleInfoCompilerPlugin.java index 31df637292..e66a475dbf 100644 --- a/java/maven/module-info-compiler-maven-plugin/src/main/java/org/apache/arrow/maven/plugins/ModuleInfoCompilerPlugin.java +++ b/java/maven/mo
(arrow) branch main updated (f0a6f49016 -> 7179511a47)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from f0a6f49016 GH-39344: [C++][FS][Azure] Support azure cli auth (#41976) add 7179511a47 GH-40820: [Java] Adding Spotless to Adapter module (#42048) No new revisions were added by this update. Summary of changes: java/adapter/avro/pom.xml | 5 + .../org/apache/arrow/adapter/avro/AvroToArrow.java | 15 +- .../arrow/adapter/avro/AvroToArrowConfig.java | 25 +- .../adapter/avro/AvroToArrowConfigBuilder.java | 20 +- .../arrow/adapter/avro/AvroToArrowUtils.java | 286 ++- .../adapter/avro/AvroToArrowVectorIterator.java| 39 +- .../adapter/avro/consumers/AvroArraysConsumer.java | 9 +- .../avro/consumers/AvroBooleanConsumer.java| 10 +- .../adapter/avro/consumers/AvroBytesConsumer.java | 10 +- .../adapter/avro/consumers/AvroDoubleConsumer.java | 10 +- .../adapter/avro/consumers/AvroEnumConsumer.java | 9 +- .../adapter/avro/consumers/AvroFixedConsumer.java | 10 +- .../adapter/avro/consumers/AvroFloatConsumer.java | 10 +- .../adapter/avro/consumers/AvroIntConsumer.java| 9 +- .../adapter/avro/consumers/AvroLongConsumer.java | 10 +- .../adapter/avro/consumers/AvroMapConsumer.java| 9 +- .../adapter/avro/consumers/AvroNullConsumer.java | 6 +- .../adapter/avro/consumers/AvroStringConsumer.java | 10 +- .../adapter/avro/consumers/AvroStructConsumer.java | 11 +- .../adapter/avro/consumers/AvroUnionsConsumer.java | 13 +- .../adapter/avro/consumers/BaseAvroConsumer.java | 3 +- .../avro/consumers/CompositeAvroConsumer.java | 15 +- .../arrow/adapter/avro/consumers/Consumer.java | 26 +- .../arrow/adapter/avro/consumers/SkipConsumer.java | 15 +- .../arrow/adapter/avro/consumers/SkipFunction.java | 6 +- .../avro/consumers/logical/AvroDateConsumer.java | 10 +- .../consumers/logical/AvroDecimalConsumer.java | 27 +- .../consumers/logical/AvroTimeMicroConsumer.java | 10 +- .../consumers/logical/AvroTimeMillisConsumer.java | 10 +- .../logical/AvroTimestampMicrosConsumer.java | 10 +- .../logical/AvroTimestampMillisConsumer.java | 10 +- .../arrow/adapter/avro/AvroLogicalTypesTest.java | 51 +- .../arrow/adapter/avro/AvroSkipFieldTest.java | 110 +++- .../apache/arrow/adapter/avro/AvroTestBase.java| 45 +- .../adapter/avro/AvroToArrowIteratorTest.java | 51 +- .../apache/arrow/adapter/avro/AvroToArrowTest.java | 58 +-- .../adapter/avro/TestWriteReadAvroRecord.java | 15 +- java/adapter/jdbc/pom.xml | 5 + .../arrow/adapter/jdbc/ArrowVectorIterator.java| 49 +- .../org/apache/arrow/adapter/jdbc/Constants.java | 9 +- .../apache/arrow/adapter/jdbc/JdbcFieldInfo.java | 74 ++- .../arrow/adapter/jdbc/JdbcParameterBinder.java| 34 +- .../org/apache/arrow/adapter/jdbc/JdbcToArrow.java | 69 +-- .../arrow/adapter/jdbc/JdbcToArrowConfig.java | 259 +- .../adapter/jdbc/JdbcToArrowConfigBuilder.java | 130 +++-- .../arrow/adapter/jdbc/JdbcToArrowUtils.java | 196 .../adapter/jdbc/binder/BaseColumnBinder.java | 2 +- .../arrow/adapter/jdbc/binder/BigIntBinder.java| 5 +- .../arrow/adapter/jdbc/binder/BitBinder.java | 5 +- .../arrow/adapter/jdbc/binder/ColumnBinder.java| 17 +- .../jdbc/binder/ColumnBinderArrowTypeVisitor.java | 109 ++-- .../arrow/adapter/jdbc/binder/DateDayBinder.java | 9 +- .../arrow/adapter/jdbc/binder/DateMilliBinder.java | 10 +- .../adapter/jdbc/binder/Decimal128Binder.java | 14 +- .../adapter/jdbc/binder/Decimal256Binder.java | 14 +- .../adapter/jdbc/binder/FixedSizeBinaryBinder.java | 15 +- .../arrow/adapter/jdbc/binder/Float4Binder.java| 9 +- .../arrow/adapter/jdbc/binder/Float8Binder.java| 12 +- .../arrow/adapter/jdbc/binder/IntBinder.java | 5 +- .../arrow/adapter/jdbc/binder/ListBinder.java | 17 +- .../arrow/adapter/jdbc/binder/MapBinder.java | 19 +- .../adapter/jdbc/binder/NullableColumnBinder.java | 9 +- .../arrow/adapter/jdbc/binder/SmallIntBinder.java | 8 +- .../arrow/adapter/jdbc/binder/Time32Binder.java| 13 +- .../arrow/adapter/jdbc/binder/Time64Binder.java| 13 +- .../arrow/adapter/jdbc/binder/TimeStampBinder.java | 24 +- .../arrow/adapter/jdbc/binder/TinyIntBinder.java | 5 +- .../arrow/adapter/jdbc/binder/VarBinaryBinder.java | 16 +- .../arrow/adapter/jdbc/binder/VarCharBinder.java | 16 +- .../arrow/adapter/jdbc/binder/package-info.java| 5 +- .../arrow/adapter/jdbc/consumer/ArrayConsumer.java | 32 +- .../arrow/adapter/jdbc/consumer/BaseConsumer.java | 3 +- .../adapter/jdbc/consumer/BigIntConsumer.java | 29 +- .../adapter/jdbc/consumer/BinaryConsumer.java | 39 +- .../arrow/adapter/jdbc/consumer/BitConsumer.java | 29
(arrow) branch main updated: MINOR: [Java] Bump org.jacoco:jacoco-maven-plugin from 0.8.11 to 0.8.12 in /java (#41516)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 95db23e9e0 MINOR: [Java] Bump org.jacoco:jacoco-maven-plugin from 0.8.11 to 0.8.12 in /java (#41516) 95db23e9e0 is described below commit 95db23e9e02c7d724269b0a0f241e53ecfbe76be Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon Jun 10 13:41:04 2024 +0900 MINOR: [Java] Bump org.jacoco:jacoco-maven-plugin from 0.8.11 to 0.8.12 in /java (#41516) Bumps [org.jacoco:jacoco-maven-plugin](https://github.com/jacoco/jacoco) from 0.8.11 to 0.8.12. Release notes Sourced from https://github.com/jacoco/jacoco/releases;>org.jacoco:jacoco-maven-plugin's releases. 0.8.12 New Features JaCoCo now officially supports Java 22 (GitHub https://redirect.github.com/jacoco/jacoco/issues/1596;>#1596). Experimental support for Java 23 class files (GitHub https://redirect.github.com/jacoco/jacoco/issues/1553;>#1553). Fixed bugs Branches added by the Kotlin compiler for functions with default arguments and having more than 32 parameters are filtered out during generation of report (GitHub https://redirect.github.com/jacoco/jacoco/issues/1556;>#1556). Branch added by the Kotlin compiler version 1.5.0 and above for reading from lateinit property is filtered out during generation of report (GitHub https://redirect.github.com/jacoco/jacoco/issues/1568;>#1568). Non-functional Changes JaCoCo now depends on ASM 9.7 (GitHub https://redirect.github.com/jacoco/jacoco/issues/1600;>#1600). Commits https://github.com/jacoco/jacoco/commit/dbfb6f2ad904158b5b40a93fea222e263aeaf9ab;>dbfb6f2 Prepare release 0.8.12 https://github.com/jacoco/jacoco/commit/a50585b9e0770c363c56d887a8f639dc05411073;>a50585b Upgrade maven-plugin-plugin to 3.6.4 (https://redirect.github.com/jacoco/jacoco/issues/1604;>#1604) https://github.com/jacoco/jacoco/commit/fd63cc5478bcd7b32e2070d93ead8c879b423841;>fd63cc5 Configure labels that Dependabot assigns to PRs (https://redirect.github.com/jacoco/jacoco/issues/1603;>#1603) https://github.com/jacoco/jacoco/commit/03a53334c1e3a28ed587feb5adbfa3c0aa536990;>03a5333 Add configuration for Dependabot to simplify updates of ASM (https://redirect.github.com/jacoco/jacoco/issues/1601;>#1601) https://github.com/jacoco/jacoco/commit/40ff9fb3b13bb484344936dde4a90b083b79cdbd;>40ff9fb Upgrade ASM to 9.7 (https://redirect.github.com/jacoco/jacoco/issues/1600;>#1600) https://github.com/jacoco/jacoco/commit/907717832435f5085b67d02e1eeec5b63b0f415b;>9077178 Happy birthday Java 22! (https://redirect.github.com/jacoco/jacoco/issues/1596;>#1596) https://github.com/jacoco/jacoco/commit/7edd1b511a174a663f633c34ae8c951a0ae77d20;>7edd1b5 Bump actions/setup-java from 4.1.0 to 4.2.1 (https://redirect.github.com/jacoco/jacoco/issues/1594;>#1594) https://github.com/jacoco/jacoco/commit/e50b547bc26ff198acfd16311c028f340af38699;>e50b547 Upgrade ECJ to 3.37.0 (https://redirect.github.com/jacoco/jacoco/issues/1590;>#1590) https://github.com/jacoco/jacoco/commit/a1144d02ff0f89d6603214676730a2d5616cf466;>a1144d0 Upgrade maven-site-plugin to 3.12.1 (https://redirect.github.com/jacoco/jacoco/issues/1586;>#1586) https://github.com/jacoco/jacoco/commit/04b0141d6ae5e6d1f00f15341b29dd4734ab778a;>04b0141 Bump actions/setup-java from 4.0.0 to 4.1.0 (https://redirect.github.com/jacoco/jacoco/issues/1587;>#1587) Additional commits viewable in https://github.com/jacoco/jacoco/compare/v0.8.11...v0.8.12;>compare view [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.jacoco:jacoco-maven-plugin=maven=0.8.11=0.8.12)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@ dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@ dependabot rebase` will rebase this PR - `@ dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@ dependabot merge` will merge this PR after your CI passes on it - `@ dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@ dependabot cancel merge` will cancel a previously requested merge and block automerging - `@ dependabot reopen` will reopen this PR if it is closed - `@ dependabot cl
(arrow) branch dependabot/maven/java/org.jacoco-jacoco-maven-plugin-0.8.12 deleted (was 5c0388628d)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.jacoco-jacoco-maven-plugin-0.8.12 in repository https://gitbox.apache.org/repos/asf/arrow.git was 5c0388628d MINOR: [Java] Bump org.jacoco:jacoco-maven-plugin in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated: GH-42042: [Java] Update Unit Tests for Compressions Module (#42044)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 7c15568aa7 GH-42042: [Java] Update Unit Tests for Compressions Module (#42044) 7c15568aa7 is described below commit 7c15568aa71c1366af5eadb6140fa445f6ce4cd0 Author: Hyunseok Seo AuthorDate: Mon Jun 10 09:48:05 2024 +0900 GH-42042: [Java] Update Unit Tests for Compressions Module (#42044) ### Rationale for this change Update package from JUnit 4(`org.junit`) to JUnit 5(`org.junit.jupiter`). ### What changes are included in this PR? - [x] Replacing `org.junit` with `org.junit.jupiter.api`. - [x] Updating `Assertions.assertXXX` to `assertXXX` using static imports - [x] Updating annotations such as `@ After`. - `@ After` -> `@ AfterEach` - [x] Doing self review ### Are these changes tested? Yes, existing tests have passed. ### Are there any user-facing changes? No. * GitHub Issue: #42042 Authored-by: Hyunseok Seo Signed-off-by: David Li --- .../TestArrowReaderWriterWithCompression.java | 59 -- 1 file changed, 31 insertions(+), 28 deletions(-) diff --git a/java/compression/src/test/java/org/apache/arrow/compression/TestArrowReaderWriterWithCompression.java b/java/compression/src/test/java/org/apache/arrow/compression/TestArrowReaderWriterWithCompression.java index af28333746..24d6abf3cb 100644 --- a/java/compression/src/test/java/org/apache/arrow/compression/TestArrowReaderWriterWithCompression.java +++ b/java/compression/src/test/java/org/apache/arrow/compression/TestArrowReaderWriterWithCompression.java @@ -17,6 +17,11 @@ package org.apache.arrow.compression; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + import java.io.ByteArrayOutputStream; import java.io.IOException; import java.nio.channels.Channels; @@ -46,9 +51,7 @@ import org.apache.arrow.vector.types.pojo.Field; import org.apache.arrow.vector.types.pojo.FieldType; import org.apache.arrow.vector.types.pojo.Schema; import org.apache.arrow.vector.util.ByteArrayReadableSeekableByteChannel; -import org.junit.After; -import org.junit.Assert; -import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -67,7 +70,7 @@ public class TestArrowReaderWriterWithCompression { root = null; } - @After + @AfterEach public void tearDown() { if (root != null) { root.close(); @@ -134,19 +137,19 @@ public class TestArrowReaderWriterWithCompression { try (ArrowFileReader reader = new ArrowFileReader(new ByteArrayReadableSeekableByteChannel(out.toByteArray()), allocator, CommonsCompressionFactory.INSTANCE)) { - Assertions.assertEquals(1, reader.getRecordBlocks().size()); - Assertions.assertTrue(reader.loadNextBatch()); - Assertions.assertTrue(root.equals(reader.getVectorSchemaRoot())); - Assertions.assertFalse(reader.loadNextBatch()); + assertEquals(1, reader.getRecordBlocks().size()); + assertTrue(reader.loadNextBatch()); + assertTrue(root.equals(reader.getVectorSchemaRoot())); + assertFalse(reader.loadNextBatch()); } // without compression try (ArrowFileReader reader = new ArrowFileReader(new ByteArrayReadableSeekableByteChannel(out.toByteArray()), allocator, NoCompressionCodec.Factory.INSTANCE)) { - Assertions.assertEquals(1, reader.getRecordBlocks().size()); - Exception exception = Assert.assertThrows(IllegalArgumentException.class, + assertEquals(1, reader.getRecordBlocks().size()); + Exception exception = assertThrows(IllegalArgumentException.class, reader::loadNextBatch); - Assertions.assertEquals("Please add arrow-compression module to use CommonsCompressionFactory for ZSTD", + assertEquals("Please add arrow-compression module to use CommonsCompressionFactory for ZSTD", exception.getMessage()); } } @@ -158,17 +161,17 @@ public class TestArrowReaderWriterWithCompression { try (ArrowStreamReader reader = new ArrowStreamReader(new ByteArrayReadableSeekableByteChannel(out.toByteArray()), allocator, CommonsCompressionFactory.INSTANCE)) { - Assert.assertTrue(reader.loadNextBatch()); - Assert.assertTrue(root.equals(reader.getVectorSchemaRoot())); - Assert.assertFalse(reader.loadNextBatch()); + assertTrue(reader.loadNextBatch()); + assertTrue(root.equals(rea
(arrow) branch main updated (a045770b94 -> fe4d04f081)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from a045770b94 GH-42006: [CI][Python] Use pip install -e instead of setup.py build_ext --inplace for installing pyarrow on verification script (#42007) add fe4d04f081 GH-42002: [Java] Update Unit Tests for Vector Module (#42019) No new revisions were added by this update. Summary of changes: .../org/apache/arrow/vector/ITTestLargeVector.java | 8 +- .../org/apache/arrow/vector/TestBitVector.java | 73 +++--- .../apache/arrow/vector/TestBitVectorHelper.java | 6 +- .../arrow/vector/TestBufferOwnershipTransfer.java | 8 +- .../java/org/apache/arrow/vector/TestCopyFrom.java | 85 +++ .../apache/arrow/vector/TestDecimal256Vector.java | 18 +- .../org/apache/arrow/vector/TestDecimalVector.java | 20 +- .../apache/arrow/vector/TestDenseUnionVector.java | 25 ++- .../apache/arrow/vector/TestDictionaryVector.java | 28 ++- .../apache/arrow/vector/TestDurationVector.java| 16 +- .../arrow/vector/TestFixedSizeBinaryVector.java| 16 +- .../arrow/vector/TestFixedSizeListVector.java | 120 +- .../vector/TestIntervalMonthDayNanoVector.java | 13 +- .../arrow/vector/TestIntervalYearVector.java | 14 +- .../apache/arrow/vector/TestLargeListVector.java | 35 ++- .../arrow/vector/TestLargeVarBinaryVector.java | 18 +- .../arrow/vector/TestLargeVarCharVector.java | 92 .../org/apache/arrow/vector/TestListVector.java| 35 ++- .../arrow/vector/TestNullCheckingForGet.java | 10 +- .../vector/TestOutOfMemoryForValueVector.java | 52 +++-- .../TestOversizedAllocationForValueVector.java | 155 ++--- .../apache/arrow/vector/TestPeriodDuration.java| 6 +- .../org/apache/arrow/vector/TestStructVector.java | 23 +- .../org/apache/arrow/vector/TestUnionVector.java | 25 ++- .../apache/arrow/vector/TestVarCharListVector.java | 17 +- .../org/apache/arrow/vector/TestVectorAlloc.java | 14 +- .../org/apache/arrow/vector/TestVectorReAlloc.java | 70 +++--- .../org/apache/arrow/vector/TestVectorReset.java | 14 +- .../apache/arrow/vector/TestVectorSchemaRoot.java | 43 ++-- .../apache/arrow/vector/TestVectorUnloadLoad.java | 35 ++- .../vector/compare/TestTypeEqualsVisitor.java | 14 +- .../vector/complex/TestDenseUnionBufferSize.java | 3 +- .../vector/complex/impl/TestComplexCopier.java | 13 +- .../vector/complex/impl/TestPromotableWriter.java | 42 ++-- .../vector/complex/writer/TestComplexWriter.java | 244 +++-- .../vector/complex/writer/TestSimpleWriter.java| 38 ++-- .../org/apache/arrow/vector/ipc/BaseFileTest.java | 209 +- .../vector/ipc/ITTestIPCWithLargeArrowBuffers.java | 10 +- .../arrow/vector/ipc/MessageSerializerTest.java| 14 +- .../org/apache/arrow/vector/ipc/TestArrowFile.java | 6 +- .../apache/arrow/vector/ipc/TestArrowFooter.java | 4 +- .../arrow/vector/ipc/TestArrowReaderWriter.java| 37 ++-- .../apache/arrow/vector/ipc/TestArrowStream.java | 15 +- .../arrow/vector/ipc/TestArrowStreamPipe.java | 20 +- .../org/apache/arrow/vector/ipc/TestJSONFile.java | 11 +- .../org/apache/arrow/vector/ipc/TestRoundTrip.java | 158 +++-- .../vector/ipc/TestUIntDictionaryRoundTrip.java| 88 .../ipc/message/TestMessageMetadataResult.java | 4 +- .../org/apache/arrow/vector/pojo/TestConvert.java | 6 +- .../vector/testing/TestValueVectorPopulator.java | 12 +- .../vector/testing/ValueVectorDataPopulator.java | 2 +- .../arrow/vector/types/pojo/TestExtensionType.java | 67 +++--- .../apache/arrow/vector/types/pojo/TestField.java | 8 +- .../apache/arrow/vector/types/pojo/TestSchema.java | 10 +- .../arrow/vector/util/DecimalUtilityTest.java | 19 +- .../vector/util/TestDataSizeRoundingUtil.java | 4 +- .../util/TestElementAddressableVectorIterator.java | 14 +- .../arrow/vector/util/TestMapWithOrdinal.java | 12 +- .../arrow/vector/util/TestMultiMapWithOrdinal.java | 43 ++-- .../arrow/vector/util/TestReusableByteArray.java | 20 +- .../apache/arrow/vector/util/TestSchemaUtil.java | 4 +- .../apache/arrow/vector/util/TestValidator.java| 6 +- .../arrow/vector/util/TestVectorAppender.java | 14 +- .../arrow/vector/util/TestVectorBatchAppender.java | 12 +- .../vector/util/TestVectorSchemaRootAppender.java | 12 +- .../arrow/vector/validate/TestValidateVector.java | 12 +- .../vector/validate/TestValidateVectorFull.java| 12 +- .../validate/TestValidateVectorSchemaRoot.java | 12 +- .../validate/TestValidateVectorTypeVisitor.java| 10 +- 69 files changed, 1191 insertions(+), 1144 deletions(-)
(arrow) branch main updated: GH-41307: [Java] Use org.apache:apache parent pom version 31 (#41772)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 01d2fa0d46 GH-41307: [Java] Use org.apache:apache parent pom version 31 (#41772) 01d2fa0d46 is described below commit 01d2fa0d461869a07b2ffeee517beb8116bd0ce2 Author: Laurent Goujon AuthorDate: Thu Jun 6 23:19:06 2024 -0700 GH-41307: [Java] Use org.apache:apache parent pom version 31 (#41772) Use/update Maven modules to `org.apache:apache:31` and clean up Maven modules to remove unnecessary configuration or outdated workarounds * Add `org.apache:apache:31` to `org.apache.arrow:arrow-bom` and `org.apache.arrow.maven.plugins:arrow-maven-plugins` to make them conformant with ASF standards * Update `org.apache.arrow:arrow-java-root` parent to `org.apache:parent:31` * Use `version.*` and other properties to override plugin versions defined by `org.apache:parent` * Move standalone plugin versions under pluginManagement at the top level * Cleanup redundant plugin version or configuration declaration * Update `maven-dependency-plugin` to 3.6.1 and add the required overrides when necessary * Update `maven-shade-plugin` to 3.5.1 (via `org.apache:parent`) - disable reduced dependency pom creation for non-terminal modules * Remove enforcer check for java and maven version (handled by `org.apache:parent`) * Remove unnecessary `mvnrepository` link comments * Remove `m2e.version` property check in profiles (only needed for errorprone plugin configuration which is incompatible with M2E) * Cleanup `argLine` overrides for surefire/failsafe plugins * Remove unnecessary `../pom.xml` `` directives * Remove source/target/encoding configuration properties for `maven-compiler-plugin`, `maven-javadoc-plugin` and `maven-resources-plugin` as it is handled by `org.apache:parent` and plugin themselves * Remove unnecessary copy of codegen templates in `arrow-vector` module * Remove unnecessary junit jupiter engine dependencies for surefire/failsafe plugins. * GitHub Issue: #41307 Lead-authored-by: Laurent Goujon Co-authored-by: Laurent Goujon Signed-off-by: David Li --- ci/scripts/java_full_build.sh | 10 +- dev/tasks/tasks.yml| 2 - java/adapter/avro/pom.xml | 9 - java/adapter/jdbc/pom.xml | 7 - java/adapter/orc/pom.xml | 17 ++ java/bom/pom.xml | 44 - java/c/pom.xml | 1 - java/flight/flight-core/pom.xml| 29 +-- java/flight/flight-integration-tests/pom.xml | 2 - java/flight/flight-sql-jdbc-core/pom.xml | 10 - java/flight/flight-sql-jdbc-driver/pom.xml | 1 - java/flight/flight-sql/pom.xml | 5 - java/format/pom.xml| 2 - java/gandiva/pom.xml | 19 +- .../module-info-compiler-maven-plugin/pom.xml | 28 +-- java/maven/pom.xml | 120 ++-- java/memory/memory-core/pom.xml| 22 +-- java/performance/pom.xml | 41 +--- java/pom.xml | 207 + java/tools/pom.xml | 22 ++- java/vector/pom.xml| 94 +- 21 files changed, 226 insertions(+), 466 deletions(-) diff --git a/ci/scripts/java_full_build.sh b/ci/scripts/java_full_build.sh index 2734f3e9db..d914aa2d84 100755 --- a/ci/scripts/java_full_build.sh +++ b/ci/scripts/java_full_build.sh @@ -49,21 +49,13 @@ fi # build the entire project mvn clean \ install \ -assembly:single \ -source:jar \ -javadoc:jar \ -Papache-release \ -Parrow-c-data \ -Parrow-jni \ -Darrow.cpp.build.dir=$dist_dir \ --Darrow.c.jni.dist.dir=$dist_dir \ --DdescriptorId=source-release +-Darrow.c.jni.dist.dir=$dist_dir # copy all jar, zip and pom files to the distribution folder -find . \ - "(" -name "*-javadoc.jar" -o -name "*-sources.jar" ")" \ - -exec echo {} ";" \ - -exec cp {} $dist_dir ";" find ~/.m2/repository/org/apache/arrow \ "(" \ -name "*.jar" -o \ diff --git a/dev/tasks/tasks.yml b/dev/tasks/tasks.yml index d8e09ec207..2d84751d0f 100644 --- a/dev/tasks/tasks.yml +++ b/dev/tasks/tasks.yml @@ -747,7 +747,6 @@ tasks: - arrow-jdbc-{no_rc_snapshot_version}.pom - arrow-maven-plugins-{no_rc_snapshot_version}-cyclonedx.json - arrow-maven-plugins-{no_rc_snapshot_version}-cyclonedx.xml - - a
(arrow-adbc) branch main updated (43ee3aa02 -> be9c2103c)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from 43ee3aa02 feat(csharp/src/Drivers/Apache): improve type name handling for CHAR/VARCHAR/DECIMAL (#1896) add be9c2103c chore: update gosnowflake dependency to 1.10.1 (#1892) No new revisions were added by this update. Summary of changes: go/adbc/go.mod | 44 ++-- go/adbc/go.sum | 48 2 files changed, 70 insertions(+), 22 deletions(-)
(arrow) branch main updated: GH-41968: [Java] Implement TransferPair functionality for BinaryView (#41980)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new cea7323772 GH-41968: [Java] Implement TransferPair functionality for BinaryView (#41980) cea7323772 is described below commit cea7323772beb5be6e5002624be36e6575a36418 Author: Vibhatha Lakmal Abeykoon AuthorDate: Thu Jun 6 13:07:11 2024 +0530 GH-41968: [Java] Implement TransferPair functionality for BinaryView (#41980) ### Rationale for this change This PR contains the transferPair functionality for BinaryView vectors. ### What changes are included in this PR? This includes the addition of transferPair functionality in `ViewVarCharBinaryVector` and corresponding test cases. ### Are these changes tested? Yes ### Are there any user-facing changes? No * GitHub Issue: #41968 Authored-by: Vibhatha Abeykoon Signed-off-by: David Li --- .../apache/arrow/vector/ViewVarBinaryVector.java | 45 +- .../apache/arrow/vector/TestSplitAndTransfer.java | 392 ++ .../apache/arrow/vector/TestVarCharViewVector.java | 814 - 3 files changed, 756 insertions(+), 495 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/ViewVarBinaryVector.java b/java/vector/src/main/java/org/apache/arrow/vector/ViewVarBinaryVector.java index 393df96b29..0a043b5106 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/ViewVarBinaryVector.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/ViewVarBinaryVector.java @@ -205,14 +205,12 @@ public final class ViewVarBinaryVector extends BaseVariableWidthViewVector { */ @Override public TransferPair getTransferPair(String ref, BufferAllocator allocator) { -// TODO: https://github.com/apache/arrow/issues/40932 -throw new UnsupportedOperationException("Unsupported operation"); +return new TransferImpl(ref, allocator); } @Override public TransferPair getTransferPair(Field field, BufferAllocator allocator) { -// TODO: https://github.com/apache/arrow/issues/40932 -throw new UnsupportedOperationException("Unsupported operation"); +return new TransferImpl(field, allocator); } /** @@ -223,7 +221,42 @@ public final class ViewVarBinaryVector extends BaseVariableWidthViewVector { */ @Override public TransferPair makeTransferPair(ValueVector to) { -// TODO: https://github.com/apache/arrow/issues/40932 -throw new UnsupportedOperationException("Unsupported operation"); +return new TransferImpl((ViewVarBinaryVector) to); + } + + private class TransferImpl implements TransferPair { +ViewVarBinaryVector to; + +public TransferImpl(String ref, BufferAllocator allocator) { + to = new ViewVarBinaryVector(ref, field.getFieldType(), allocator); +} + +public TransferImpl(Field field, BufferAllocator allocator) { + to = new ViewVarBinaryVector(field, allocator); +} + +public TransferImpl(ViewVarBinaryVector to) { + this.to = to; +} + +@Override +public ViewVarBinaryVector getTo() { + return to; +} + +@Override +public void transfer() { + transferTo(to); +} + +@Override +public void splitAndTransfer(int startIndex, int length) { + splitAndTransferTo(startIndex, length, to); +} + +@Override +public void copyValueSafe(int fromIndex, int toIndex) { + to.copyFromSafe(fromIndex, toIndex, ViewVarBinaryVector.this); +} } } diff --git a/java/vector/src/test/java/org/apache/arrow/vector/TestSplitAndTransfer.java b/java/vector/src/test/java/org/apache/arrow/vector/TestSplitAndTransfer.java index d2c03930ca..fece93de9b 100644 --- a/java/vector/src/test/java/org/apache/arrow/vector/TestSplitAndTransfer.java +++ b/java/vector/src/test/java/org/apache/arrow/vector/TestSplitAndTransfer.java @@ -67,7 +67,8 @@ public class TestSplitAndTransfer { vector.setValueCount(valueCount); } - private void populateViewVarcharVector(final ViewVarCharVector vector, int valueCount, String[] compareArray) { + private void populateBaseVariableWidthViewVector(final BaseVariableWidthViewVector vector, int valueCount, + String[] compareArray) { for (int i = 0; i < valueCount; i += 3) { final String s = String.format("%010d", i); vector.set(i, s.getBytes(StandardCharsets.UTF_8)); @@ -120,11 +121,16 @@ public class TestSplitAndTransfer { transferPair = varCharVector.getTransferPair(allocator); transferPair.splitAndTransfer(0, 0); assertEquals(0, transferPair.getTo().getValueCount()); -// BaseVariableWidthViewVector +// BaseVariableWidthViewVector: ViewVarCharVector ViewVarCharVector viewVarCharVector = new ViewVarCharVector("", allocator);
(arrow) branch dependabot/maven/java/com.github.luben-zstd-jni-1.5.6-3 deleted (was 40e55ab282)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/com.github.luben-zstd-jni-1.5.6-3 in repository https://gitbox.apache.org/repos/asf/arrow.git was 40e55ab282 MINOR: [Java] Bump com.github.luben:zstd-jni in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch dependabot/maven/java/org.apache.calcite.avatica-avatica-1.25.0 deleted (was 519f7842bd)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.apache.calcite.avatica-avatica-1.25.0 in repository https://gitbox.apache.org/repos/asf/arrow.git was 519f7842bd MINOR: [Java] Bump org.apache.calcite.avatica:avatica in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated (41810749e0 -> 8b2c7e2893)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 41810749e0 MINOR: [Java] Bump org.apache.calcite.avatica:avatica from 1.24.0 to 1.25.0 in /java (#41212) add 8b2c7e2893 MINOR: [Java] Bump com.github.luben:zstd-jni from 1.5.5-11 to 1.5.6-3 in /java (#41521) No new revisions were added by this update. Summary of changes: java/compression/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow) branch main updated: MINOR: [Java] Bump org.apache.calcite.avatica:avatica from 1.24.0 to 1.25.0 in /java (#41212)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 41810749e0 MINOR: [Java] Bump org.apache.calcite.avatica:avatica from 1.24.0 to 1.25.0 in /java (#41212) 41810749e0 is described below commit 41810749e086278aea541240a16e9cf1e32eab80 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Jun 6 16:15:52 2024 +0900 MINOR: [Java] Bump org.apache.calcite.avatica:avatica from 1.24.0 to 1.25.0 in /java (#41212) Bumps [org.apache.calcite.avatica:avatica](https://github.com/apache/calcite-avatica) from 1.24.0 to 1.25.0. Commits https://github.com/apache/calcite-avatica/commit/62b0fdd3f4e0173d8492eb0e203056f0938dd6f5;>62b0fdd [CALCITE-6334] Release Avatica 1.25.0 https://github.com/apache/calcite-avatica/commit/c0cb4b71f187e688e3dfd579111c3c646b9c47af;>c0cb4b7 [CALCITE-6248] Illegal dates are accepted by casts https://github.com/apache/calcite-avatica/commit/8c36e0196d10574280518d406f288a3bd83ae4dc;>8c36e01 [CALCITE-6282] Avatica ignores time precision when returning TIME results https://github.com/apache/calcite-avatica/commit/ef9a5a653411fee2eb94505dc6bb2fd437cb76d5;>ef9a5a6 [CALCITE-6137] Upgrade Gradle from 8.1.1 to 8.4, support jdk21 https://github.com/apache/calcite-avatica/commit/c12c3a3305f6435fedf13e6ce6867c30c21ffd58;>c12c3a3 Apply same vcs.xml as for Calcite https://github.com/apache/calcite-avatica/commit/4b9c8231b923fa7d7ea57131f0c5cb1dd2099613;>4b9c823 [CALCITE-6209] Long queries are failing with java.net.SocketTimeoutException... https://github.com/apache/calcite-avatica/commit/275a082d364e00a9f2e9d50a4468fef0782e3c81;>275a082 [CALCITE-6280] Jetty version number leaked by Avatica http server https://github.com/apache/calcite-avatica/commit/bc7ba9e9ad3eeddad573f484039efd3563951780;>bc7ba9e Disable JIRA worklog notifications for GitHub PRs https://github.com/apache/calcite-avatica/commit/178ff82caeb415d38c5852b8f6c7373bd8d97718;>178ff82 Add Calcite CLI tool to list of Avatica Clients on website https://github.com/apache/calcite-avatica/commit/07c6b8d92f478646e3e605a80dc0d60329c37d33;>07c6b8d [CALCITE-6212] Config locale = en_US for javadoc task Additional commits viewable in https://github.com/apache/calcite-avatica/compare/rel/avatica-1.24.0...rel/avatica-1.25.0;>compare view [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.calcite.avatica:avatica=maven=1.24.0=1.25.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@ dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@ dependabot rebase` will rebase this PR - `@ dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@ dependabot merge` will merge this PR after your CI passes on it - `@ dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@ dependabot cancel merge` will cancel a previously requested merge and block automerging - `@ dependabot reopen` will reopen this PR if it is closed - `@ dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@ dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@ dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@ dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@ dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Signed-off-by: David Li --- java/flight/flight-sql-jdbc-core/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java/flight/flight-sql-jdbc-core/pom.xml b/java/flight/flight-sql-jdbc-core/pom.xml index fbab69df3b..459412e0f8 100644 --- a/java/fl
(arrow) branch main updated (3d11205517 -> dbcce63b92)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 3d11205517 GH-41999: [Swift] Add methods for adding array and vargs to arrow array (#42000) add dbcce63b92 MINOR: [Java] Bump com.google.errorprone:error_prone_core from 2.10.0 to 2.28.0 in /java - Manual Fix (#41996) No new revisions were added by this update. Summary of changes: java/pom.xml | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-)
(arrow) branch dependabot/maven/java/com.google.protobuf-protobuf-java-4.27.0 deleted (was e94a80f34d)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/com.google.protobuf-protobuf-java-4.27.0 in repository https://gitbox.apache.org/repos/asf/arrow.git was e94a80f34d MINOR: [Java] Bump com.google.protobuf:protobuf-java in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch dependabot/maven/java/org.apache-apache-32 deleted (was 2c7f09dd30)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.apache-apache-32 in repository https://gitbox.apache.org/repos/asf/arrow.git was 2c7f09dd30 MINOR: [Java] Bump org.apache:apache from 18 to 32 in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated (7bc2452b35 -> 524a463207)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 7bc2452b35 GH-41902: [Java] Variadic Buffer Counts Incorrect (#41930) add 524a463207 GH-39649: [Java][CI] Fix or suppress spurious errorprone warnings stage 2 (#39777) No new revisions were added by this update. Summary of changes: docs/source/developers/java/development.rst| 16 + .../arrow/adapter/avro/AvroToArrowUtils.java | 28 +++--- .../adapter/avro/AvroToArrowVectorIterator.java| 2 ++ .../arrow/adapter/avro/consumers/Consumer.java | 1 + .../arrow/adapter/avro/AvroSkipFieldTest.java | 9 +++ .../apache/arrow/adapter/avro/AvroTestBase.java| 4 ++-- .../adapter/avro/AvroToArrowIteratorTest.java | 4 ++-- .../apache/arrow/adapter/avro/AvroToArrowTest.java | 2 -- .../adapter/jdbc/consumer/BinaryConsumer.java | 2 +- .../arrow/adapter/jdbc/consumer/ClobConsumer.java | 8 +++ .../arrow/adapter/jdbc/consumer/JdbcConsumer.java | 1 + .../adapter/jdbc/JdbcParameterBinderTest.java | 10 .../jdbc/JdbcToArrowCommentMetadataTest.java | 9 --- .../arrow/adapter/jdbc/JdbcToArrowConfigTest.java | 8 +++ .../arrow/adapter/jdbc/JdbcToArrowTestHelper.java | 7 -- .../arrow/adapter/jdbc/ResultSetUtility.java | 23 +++--- .../arrow/adapter/jdbc/UnreliableMetaDataTest.java | 3 --- .../adapter/jdbc/h2/JdbcAliasToArrowTest.java | 4 ++-- .../adapter/jdbc/h2/JdbcToArrowArrayTest.java | 3 ++- .../adapter/jdbc/h2/JdbcToArrowCharSetTest.java| 3 +++ .../adapter/jdbc/h2/JdbcToArrowDataTypesTest.java | 2 ++ .../jdbc/h2/JdbcToArrowMapDataTypeTest.java| 2 ++ .../arrow/adapter/jdbc/h2/JdbcToArrowNullTest.java | 2 ++ .../jdbc/h2/JdbcToArrowOptionalColumnsTest.java| 2 ++ .../arrow/adapter/jdbc/h2/JdbcToArrowTest.java | 2 ++ .../adapter/jdbc/h2/JdbcToArrowTimeZoneTest.java | 2 ++ .../c/src/main/java/org/apache/arrow/c/Format.java | 3 ++- .../org/apache/arrow/c/jni/CDataJniException.java | 4 ++-- .../java/org/apache/arrow/c/DictionaryTest.java| 25 +-- .../java/org/apache/arrow/c/NativeUtilTest.java| 2 +- .../java/org/apache/arrow/c/RoundtripTest.java | 4 ++-- .../arrow/memory/AllocationOutcomeDetails.java | 18 +++--- .../apache/arrow/memory/AllocationReservation.java | 2 +- .../org/apache/arrow/memory/BaseAllocator.java | 2 ++ .../java/org/apache/arrow/memory/BufferLedger.java | 3 ++- .../org/apache/arrow/memory/BufferManager.java | 2 +- .../org/apache/arrow/memory/CheckAllocator.java| 1 + .../org/apache/arrow/memory/ChildAllocator.java| 2 +- .../arrow/memory/LowCostIdentityHashMap.java | 3 +-- .../memory/rounding/DefaultRoundingPolicy.java | 4 .../apache/arrow/memory/util/ArrowBufPointer.java | 3 ++- .../java/org/apache/arrow/memory/util/Float16.java | 4 ++-- .../apache/arrow/memory/util/HistoricalLog.java| 11 +++-- .../org/apache/arrow/memory/util/MemoryUtil.java | 1 + .../arrow/memory/util/hash/MurmurHasher.java | 2 +- .../arrow/memory/util/hash/SimpleHasher.java | 1 + .../java/org/apache/arrow/memory/TestArrowBuf.java | 2 +- .../org/apache/arrow/memory/TestBaseAllocator.java | 4 ++-- .../arrow/memory/TestLowCostIdentityHashMap.java | 2 +- .../arrow/memory/util/TestArrowBufPointer.java | 20 .../arrow/memory/util/TestByteFunctionHelpers.java | 6 +++-- .../arrow/memory/util/hash/TestArrowBufHasher.java | 12 +- .../main/java/io/netty/buffer/NettyArrowBuf.java | 2 +- .../io/netty/buffer/UnsafeDirectLittleEndian.java | 19 +-- .../netty/buffer/TestUnsafeDirectLittleEndian.java | 3 +-- .../arrow/memory/netty/NettyAllocationManager.java | 4 +--- .../apache/arrow/memory/netty/TestEndianness.java | 16 ++--- .../arrow/memory/netty/TestNettyAllocator.java | 1 + java/pom.xml | 4 +++- 59 files changed, 187 insertions(+), 164 deletions(-)
(arrow) branch main updated: GH-41902: [Java] Variadic Buffer Counts Incorrect (#41930)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 7bc2452b35 GH-41902: [Java] Variadic Buffer Counts Incorrect (#41930) 7bc2452b35 is described below commit 7bc2452b350867b3ddc9de9ceceeef0e4d722941 Author: Vibhatha Lakmal Abeykoon AuthorDate: Tue Jun 4 12:35:13 2024 +0530 GH-41902: [Java] Variadic Buffer Counts Incorrect (#41930) ### Rationale for this change In the initial PR for `variadicBufferCounts` addition to Java spec, the non variadic buffer-ed vectors were assigned with 0 valued non-empty `variadicBufferCounts`. And this caused CIs to fail in Arrow Rust. ### What changes are included in this PR? This PR changes such that non variadic buffer-ed vectors would contain an empty `variadicBufferCounts` attribute in `ArrowRecordBatch` interface in Java. Also this includes upgrade to JUNIT5. ### Are these changes tested? Yes, from existing tests and a new test added. ### Are there any user-facing changes? No * GitHub Issue: #41902 Authored-by: Vibhatha Abeykoon Signed-off-by: David Li --- .../org/apache/arrow/c/StructVectorLoader.java | 29 +--- .../org/apache/arrow/c/StructVectorUnloader.java | 5 +- .../java/org/apache/arrow/c/DictionaryTest.java| 60 .../java/org/apache/arrow/vector/VectorLoader.java | 20 -- .../org/apache/arrow/vector/VectorUnloader.java| 5 +- .../org/apache/arrow/vector/TestValueVector.java | 56 +++ .../apache/arrow/vector/TestVarCharViewVector.java | 80 ++ 7 files changed, 238 insertions(+), 17 deletions(-) diff --git a/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java b/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java index 27acf84d30..1b0c59163a 100644 --- a/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java +++ b/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java @@ -27,6 +27,7 @@ import java.util.List; import org.apache.arrow.memory.ArrowBuf; import org.apache.arrow.memory.BufferAllocator; import org.apache.arrow.util.Collections2; +import org.apache.arrow.vector.BaseVariableWidthViewVector; import org.apache.arrow.vector.FieldVector; import org.apache.arrow.vector.TypeLayout; import org.apache.arrow.vector.complex.StructVector; @@ -54,7 +55,14 @@ public class StructVectorLoader { /** * Construct with a schema. - * + * + * The schema referred to here can be obtained from the struct vector. + * The schema here should be the children of a struct vector, not a schema + * containing the struct field itself. + * For example: + * + * Schema schema = new Schema(structVector.getField().getChildren()); + * * @param schema buffers are added based on schema. */ public StructVectorLoader(Schema schema) { @@ -90,7 +98,7 @@ public class StructVectorLoader { .fromCompressionType(recordBatch.getBodyCompression().getCodec()); decompressionNeeded = codecType != CompressionUtil.CodecType.NO_COMPRESSION; CompressionCodec codec = decompressionNeeded ? factory.createCodec(codecType) : NoCompressionCodec.INSTANCE; -Iterator variadicBufferCounts = null; +Iterator variadicBufferCounts = Collections.emptyIterator(); if (recordBatch.getVariadicBufferCounts() != null && !recordBatch.getVariadicBufferCounts().isEmpty()) { variadicBufferCounts = recordBatch.getVariadicBufferCounts().iterator(); } @@ -98,9 +106,10 @@ public class StructVectorLoader { loadBuffers(fieldVector, fieldVector.getField(), buffers, nodes, codec, variadicBufferCounts); } result.loadFieldBuffers(new ArrowFieldNode(recordBatch.getLength(), 0), Collections.singletonList(null)); -if (nodes.hasNext() || buffers.hasNext()) { - throw new IllegalArgumentException("not all nodes and buffers were consumed. nodes: " + -Collections2.toList(nodes).toString() + " buffers: " + Collections2.toList(buffers).toString()); +if (nodes.hasNext() || buffers.hasNext() || variadicBufferCounts.hasNext()) { + throw new IllegalArgumentException("not all nodes, buffers and variadicBufferCounts were consumed. nodes: " + +Collections2.toString(nodes) + " buffers: " + Collections2.toString(buffers) + " variadicBufferCounts: " + + Collections2.toString(variadicBufferCounts)); } return result; } @@ -109,10 +118,14 @@ public class StructVectorLoader { CompressionCodec codec, Iterator variadicBufferCounts) { checkArgument(nodes.hasNext(), "no more field nodes for field %s and vector %s", field, vector); ArrowFieldNode fieldNode = nodes.next(); -// variadicBufferLayoutCou
(arrow) branch main updated (813fe25967 -> 1598782d4f)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 813fe25967 GH-41829: [R] Update relative URLs in README to absolute paths to prevent CRAN check failures (#41830) add 1598782d4f GH-41262: [Java][FlightSQL] Implement stateless prepared statements (#41237) No new revisions were added by this update. Summary of changes: .../apache/arrow/flight/sql/FlightSqlClient.java | 27 ++- .../example/DoPutPreparedStatementResultPOJO.java} | 23 +- .../arrow/flight/sql/example/FlightSqlExample.java | 60 -- .../sql/example/FlightSqlStatelessExample.java | 238 + .../arrow/flight/sql/test/TestFlightSql.java | 63 +++--- .../flight/sql/test/TestFlightSqlStateless.java| 99 + 6 files changed, 449 insertions(+), 61 deletions(-) copy java/{gandiva/src/main/java/org/apache/arrow/gandiva/exceptions/GandivaException.java => flight/flight-sql/src/test/java/org/apache/arrow/flight/sql/example/DoPutPreparedStatementResultPOJO.java} (64%) create mode 100644 java/flight/flight-sql/src/test/java/org/apache/arrow/flight/sql/example/FlightSqlStatelessExample.java create mode 100644 java/flight/flight-sql/src/test/java/org/apache/arrow/flight/sql/test/TestFlightSqlStateless.java
(arrow) branch main updated (02585cd112 -> 54bece3d4c)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 02585cd112 GH-39345: [C++][FS][Azure] Add support for environment credential (#41715) add 54bece3d4c GH-41648: [Java] Memory Leak about splitAndTransfer (#41898) No new revisions were added by this update. Summary of changes: .../apache/arrow/vector/BaseFixedWidthVector.java | 4 +- .../org/apache/arrow/vector/TestValueVector.java | 197 + 2 files changed, 122 insertions(+), 79 deletions(-)
(arrow) branch main updated (6c15eb8e84 -> 706b3e09e1)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 6c15eb8e84 MINOR: [Java] Update develocity access key environment variable (#41880) add 706b3e09e1 GH-40932: [Java] Implement TransferPair functionality for StringView (#41861) No new revisions were added by this update. Summary of changes: .../arrow/vector/BaseVariableWidthViewVector.java | 166 ++- .../org/apache/arrow/vector/ViewVarCharVector.java | 52 ++- .../apache/arrow/vector/TestSplitAndTransfer.java | 250 ++- .../apache/arrow/vector/TestVarCharViewVector.java | 498 + 4 files changed, 940 insertions(+), 26 deletions(-)
(arrow) branch main updated (0b5c53ba0f -> 13630c7a83)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 0b5c53ba0f MINOR: [Java] Fix develocity cache directory name in .gitignore (#41866) add 13630c7a83 MINOR: [Java] Bump org.apache.commons:commons-compress from 1.26.0 to 1.26.2 in /java (#41853) No new revisions were added by this update. Summary of changes: java/compression/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow) branch main updated (235608beb6 -> 0b5c53ba0f)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 235608beb6 MINOR: [C++] Slight improvement for ArrayData device_type (#41814) add 0b5c53ba0f MINOR: [Java] Fix develocity cache directory name in .gitignore (#41866) No new revisions were added by this update. Summary of changes: .gitignore | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow) branch dependabot/maven/java/org.apache.maven.plugins-maven-shade-plugin-3.5.3 deleted (was 88a5684981)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.apache.maven.plugins-maven-shade-plugin-3.5.3 in repository https://gitbox.apache.org/repos/asf/arrow.git was 88a5684981 MINOR: [Java] Bump org.apache.maven.plugins:maven-shade-plugin in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch dependabot/maven/java/checker.framework.version-3.43.0 deleted (was 04c3c04140)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/checker.framework.version-3.43.0 in repository https://gitbox.apache.org/repos/asf/arrow.git was 04c3c04140 MINOR: [Java] Bump checker.framework.version in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated: MINOR: [Java] Bump checker.framework.version from 3.42.0 to 3.43.0 in /java (#41520)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new ad711ec459 MINOR: [Java] Bump checker.framework.version from 3.42.0 to 3.43.0 in /java (#41520) ad711ec459 is described below commit ad711ec4590170ad225f2cbb7b6a1113a87c6e67 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Sat May 25 11:06:19 2024 +0900 MINOR: [Java] Bump checker.framework.version from 3.42.0 to 3.43.0 in /java (#41520) Bumps `checker.framework.version` from 3.42.0 to 3.43.0. Updates `org.checkerframework:checker-qual` from 3.42.0 to 3.43.0 Release notes Sourced from https://github.com/typetools/checker-framework/releases;>org.checkerframework:checker-qual's releases. Checker Framework 3.43.0 Version 3.43.0 (May 1, 2024) User-visible changes: Method, constructor, lambda, and method reference type inference has been greatly improved. The -AconservativeUninferredTypeArguments option is no longer necessary and has been removed. Renamed command-line arguments: -AskipDirs has been renamed to -AskipFiles. -AskipDirs will continue to work for the time being. New command-line arguments: -AonlyFiles complements -AskipFiles A specialized inference algorithm for the Resource Leak Checker runs automatically as part of whole-program inference. Implementation details: Deprecated ObjectCreationNode#getConstructor in favor of new ObjectCreationNode#getTypeToInstantiate(). Renamed AbstractCFGVisualizer.visualizeBlockHelper() to visualizeBlockWithSeparator(). Moved methods from TreeUtils to subclasses of TreeUtilsAfterJava11: isConstantCaseLabelTree isDefaultCaseLabelTree isPatternCaseLabelTree Renamed BaseTypeVisitor.checkForPolymorphicQualifiers() to warnInvalidPolymorphicQualifier(). Closed issues: https://redirect.github.com/typetools/checker-framework/issues/979;>#979, https://redirect.github.com/typetools/checker-framework/issues/4559;>#4559, https://redirect.github.com/typetools/checker-framework/issues/4593;>#4593, https://redirect.github.com/typetools/checker-framework/issues/5058;>#5058, https://redirect.github.com/typetools/checker-framework/issues/5734;>#5734, https://redirect.github.com/typeto [...] Changelog Sourced from https://github.com/typetools/checker-framework/blob/master/docs/CHANGELOG.md;>org.checkerframework:checker-qual's changelog. Version 3.43.0 (May 1, 2024) User-visible changes: Method, constructor, lambda, and method reference type inference has been greatly improved. The -AconservativeUninferredTypeArguments option is no longer necessary and has been removed. Renamed command-line arguments: -AskipDirs has been renamed to -AskipFiles. -AskipDirs will continue to work for the time being. New command-line arguments: -AonlyFiles complements -AskipFiles A specialized inference algorithm for the Resource Leak Checker runs automatically as part of whole-program inference. Implementation details: Deprecated ObjectCreationNode#getConstructor in favor of new ObjectCreationNode#getTypeToInstantiate(). Renamed AbstractCFGVisualizer.visualizeBlockHelper() to visualizeBlockWithSeparator(). Moved methods from TreeUtils to subclasses of TreeUtilsAfterJava11: isConstantCaseLabelTree isDefaultCaseLabelTree isPatternCaseLabelTree Renamed BaseTypeVisitor.checkForPolymorphicQualifiers() to warnInvalidPolymorphicQualifier(). Closed issues: https://redirect.github.com/typetools/checker-framework/issues/979;>#979, https://redirect.github.com/typetools/checker-framework/issues/4559;>#4559, https://redirect.github.com/typetools/checker-framework/issues/4593;>#4593, https://redirect.github.com/typetools/checker-framework/issues/5058;>#5058, https://redirect.github.com/typetools/checker-framework/issues/5734;>#5734, https://redirect.github.com/typeto [...] https://redirect.github.com/typetools/checker-framework/issues/6322;>#6322, https://redirect.github.com/typetools/checker-framework/issues/6346;>#6346, https://redirect.github.com/typetools/checker-framework/issues/6373;>#6373, https://redirect.github.com/typetools/checker-framework/issues/6376;>#6376, https://redirect.github.com/typetools/checker-framework/issues/6378;>#6378, https://redirect.github.com/typetoo [...] https://redirect.github.com/typetools/checker-framework/issues/6406;>#6406, https://redirect.github.com/typetools/checker-framework/issues/6407;>#6407, https://redirect.github.com/typetools/checker-framework/issues/6417;>#6417, https://redirect
(arrow) branch main updated: GH-40933: [Java] Enhance the copyFrom* functionality in StringView (#41752)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 54dfb82a40 GH-40933: [Java] Enhance the copyFrom* functionality in StringView (#41752) 54dfb82a40 is described below commit 54dfb82a401fa4c1e53fccb2f152b04f7bb85f4d Author: Vibhatha Lakmal Abeykoon AuthorDate: Sat May 25 07:25:11 2024 +0530 GH-40933: [Java] Enhance the copyFrom* functionality in StringView (#41752) ### Rationale for this change Initial implementation of StringView doesn't contain `copy` functionality. This PR adds that feature. ### What changes are included in this PR? This PR adds `copyFrom` and `copyFromSafe` functions to `BaseVariableWidthViewVector`. ### Are these changes tested? Yes ### Are there any user-facing changes? No * GitHub Issue: #40933 Lead-authored-by: Vibhatha Abeykoon Co-authored-by: Vibhatha Lakmal Abeykoon Signed-off-by: David Li --- .../arrow/vector/BaseVariableWidthViewVector.java | 52 +- .../java/org/apache/arrow/vector/types/Types.java | 2 +- .../apache/arrow/vector/TestVarCharViewVector.java | 197 + 3 files changed, 243 insertions(+), 8 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java index b3e86fab05..aaa8098b69 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java @@ -33,6 +33,7 @@ import org.apache.arrow.memory.util.ArrowBufPointer; import org.apache.arrow.memory.util.ByteFunctionHelpers; import org.apache.arrow.memory.util.CommonUtil; import org.apache.arrow.memory.util.hash.ArrowBufHasher; +import org.apache.arrow.util.Preconditions; import org.apache.arrow.vector.compare.VectorVisitor; import org.apache.arrow.vector.ipc.message.ArrowFieldNode; import org.apache.arrow.vector.types.pojo.Field; @@ -1334,30 +1335,67 @@ public abstract class BaseVariableWidthViewVector extends BaseValueVector implem /** * Copy a cell value from a particular index in source vector to a particular position in this * vector. - * TODO: Improve functionality to support copying views. - * https://github.com/apache/arrow/issues/40933;>Enhance CopyFrom - * * @param fromIndex position to copy from in source vector * @param thisIndex position to copy to in this vector * @param from source vector */ @Override public void copyFrom(int fromIndex, int thisIndex, ValueVector from) { -throw new UnsupportedOperationException("copyFrom is not supported for VariableWidthVector"); +Preconditions.checkArgument(getMinorType() == from.getMinorType()); +if (from.isNull(fromIndex)) { + BitVectorHelper.unsetBit(validityBuffer, thisIndex); +} else { + final int viewLength = from.getDataBuffer().getInt((long) fromIndex * ELEMENT_SIZE); + BitVectorHelper.setBit(validityBuffer, thisIndex); + final int start = thisIndex * ELEMENT_SIZE; + final int copyStart = fromIndex * ELEMENT_SIZE; + from.getDataBuffer().getBytes(start, viewBuffer, copyStart, ELEMENT_SIZE); + if (viewLength > INLINE_SIZE) { +final int bufIndex = from.getDataBuffer().getInt(((long) fromIndex * ELEMENT_SIZE) + +LENGTH_WIDTH + PREFIX_WIDTH); +final int dataOffset = from.getDataBuffer().getInt(((long) fromIndex * ELEMENT_SIZE) + +LENGTH_WIDTH + PREFIX_WIDTH + BUF_INDEX_WIDTH); +final ArrowBuf dataBuf = ((BaseVariableWidthViewVector) from).dataBuffers.get(bufIndex); +final ArrowBuf thisDataBuf = allocateOrGetLastDataBuffer(viewLength); +thisDataBuf.setBytes(thisDataBuf.writerIndex(), dataBuf, dataOffset, viewLength); +thisDataBuf.writerIndex(thisDataBuf.writerIndex() + viewLength); + } +} +lastSet = thisIndex; } /** * Same as {@link #copyFrom(int, int, ValueVector)} except that it handles the case when the * capacity of the vector needs to be expanded before copy. - * TODO: Improve functionality to support copying views. - * https://github.com/apache/arrow/issues/40933;>Enhance CopyFrom * @param fromIndex position to copy from in source vector * @param thisIndex position to copy to in this vector * @param from source vector */ @Override public void copyFromSafe(int fromIndex, int thisIndex, ValueVector from) { -throw new UnsupportedOperationException("copyFromSafe is not supported for VariableWidthVector"); +Preconditions.checkArgument(getMinorType() == from.getMinorType()); +if (from.isNull(fromIndex)) { +
(arrow) branch main updated: GH-41730: [Java] Adding variadicBufferCounts to RecordBatch (#41732)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 0c96be3590 GH-41730: [Java] Adding variadicBufferCounts to RecordBatch (#41732) 0c96be3590 is described below commit 0c96be359084e727b82482d69f58aa79199d703d Author: Vibhatha Lakmal Abeykoon AuthorDate: Fri May 24 09:32:32 2024 +0530 GH-41730: [Java] Adding variadicBufferCounts to RecordBatch (#41732) ### Rationale for this change This PR adds the `variadicBufferCounts` attribute to `ArrowRecordBatch` in Java module. Furthermore, it also updates the `TypeLayout` functions `getTypeBufferCount` and `getTypeLayout` functions along with the corresponding test cases. Previously these changes were listed as issues https://github.com/apache/arrow/issues/40934, https://github.com/apache/arrow/issues/40935 and https://github.com/apache/arrow/issues/40931. These two tickets will also be closed by this PR. ### What changes are included in this PR? The introduced two functions to `TypeLayout` is deprecating the old API and adds a new API. In this PR we are updating a few modules to use the new API. Corresponding tests for the changed functions have also been added. This also updates the usage of `ArrowRecordBatch` across other modules and `TypeLayout` usage across a few modules. Some modules were excluded as mentioned in the issues non-goals section to be completed in a follow up effort as the scope and required tasks remain at large. These modules will still use the deprecated API for TypeLayouts, but documented in the code for updating to the new API in a follow up effort. ### Closing Subtasks - [X] https://github.com/apache/arrow/issues/40934 - [X] https://github.com/apache/arrow/issues/40935 - [X] https://github.com/apache/arrow/issues/40931 ### Are these changes tested? The changes are tested using existing tests and new tests ### Are there any user-facing changes? Yes **This PR includes breaking changes to public APIs.** * GitHub Issue: #41730 Lead-authored-by: Vibhatha Lakmal Abeykoon Co-authored-by: Vibhatha Abeykoon Signed-off-by: David Li --- .../org/apache/arrow/c/StructVectorLoader.java | 17 ++- .../org/apache/arrow/c/StructVectorUnloader.java | 22 +++- .../arrow/vector/BaseVariableWidthViewVector.java | 19 +++- .../java/org/apache/arrow/vector/TypeLayout.java | 22 ++-- .../java/org/apache/arrow/vector/VectorLoader.java | 19 +++- .../org/apache/arrow/vector/VectorUnloader.java| 23 +++- .../org/apache/arrow/vector/ViewVarCharVector.java | 2 +- .../apache/arrow/vector/ipc/JsonFileReader.java| 1 + .../apache/arrow/vector/ipc/JsonFileWriter.java| 1 + .../arrow/vector/ipc/message/ArrowRecordBatch.java | 89 ++- .../validate/ValidateVectorBufferVisitor.java | 1 + .../org/apache/arrow/vector/TestTypeLayout.java| 122 + .../apache/arrow/vector/TestVarCharViewVector.java | 66 +++ 13 files changed, 343 insertions(+), 61 deletions(-) diff --git a/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java b/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java index d9afd0189d..27acf84d30 100644 --- a/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java +++ b/java/c/src/main/java/org/apache/arrow/c/StructVectorLoader.java @@ -90,8 +90,12 @@ public class StructVectorLoader { .fromCompressionType(recordBatch.getBodyCompression().getCodec()); decompressionNeeded = codecType != CompressionUtil.CodecType.NO_COMPRESSION; CompressionCodec codec = decompressionNeeded ? factory.createCodec(codecType) : NoCompressionCodec.INSTANCE; +Iterator variadicBufferCounts = null; +if (recordBatch.getVariadicBufferCounts() != null && !recordBatch.getVariadicBufferCounts().isEmpty()) { + variadicBufferCounts = recordBatch.getVariadicBufferCounts().iterator(); +} for (FieldVector fieldVector : result.getChildrenFromFields()) { - loadBuffers(fieldVector, fieldVector.getField(), buffers, nodes, codec); + loadBuffers(fieldVector, fieldVector.getField(), buffers, nodes, codec, variadicBufferCounts); } result.loadFieldBuffers(new ArrowFieldNode(recordBatch.getLength(), 0), Collections.singletonList(null)); if (nodes.hasNext() || buffers.hasNext()) { @@ -102,10 +106,15 @@ public class StructVectorLoader { } private void loadBuffers(FieldVector vector, Field field, Iterator buffers, Iterator nodes, - CompressionCodec codec) { + CompressionCodec codec, Iterator variadicBufferCounts) { checkArgument(nodes.hasNext(), "no more field nodes for field %s and vector %s", field, vector); ArrowFieldNode fiel
(arrow) branch main updated (4ace0a18cd -> b40d368019)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 4ace0a18cd GH-41799: [Java] Migrate to com.gradle:develocity-maven-extension (#41800) add b40d368019 GH-41686: [Java] Nullability of struct child vectors not preserved in TransferPair (#41785) No new revisions were added by this update. Summary of changes: .../src/main/codegen/templates/StructWriters.java | 1 + .../org/apache/arrow/vector/TestMapVector.java | 58 +++--- 2 files changed, 40 insertions(+), 19 deletions(-)
(arrow) branch main updated (8a76082e3a -> 4ace0a18cd)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 8a76082e3a GH-40930: [Java] Implement a function to retrieve reference buffers in StringView (#41796) add 4ace0a18cd GH-41799: [Java] Migrate to com.gradle:develocity-maven-extension (#41800) No new revisions were added by this update. Summary of changes: .gitignore | 2 +- .mvn/{gradle-enterprise.xml => develocity.xml} | 10 +- .mvn/extensions.xml | 6 +++--- java/maven/module-info-compiler-maven-plugin/pom.xml | 6 +++--- java/pom.xml | 6 +++--- 5 files changed, 15 insertions(+), 15 deletions(-) rename .mvn/{gradle-enterprise.xml => develocity.xml} (78%)
(arrow) branch main updated: GH-40930: [Java] Implement a function to retrieve reference buffers in StringView (#41796)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 8a76082e3a GH-40930: [Java] Implement a function to retrieve reference buffers in StringView (#41796) 8a76082e3a is described below commit 8a76082e3a4b31ba74063093a3f279726625e245 Author: Vibhatha Lakmal Abeykoon AuthorDate: Fri May 24 09:28:19 2024 +0530 GH-40930: [Java] Implement a function to retrieve reference buffers in StringView (#41796) ### Rationale for this change This PR includes a minor changes to the `getBuffers` method in `BaseVariableWidthViewVector`. ### What changes are included in this PR? Previously the fixed buffers were the only ones returned from this method because of lack of clarity in the initial implementation stage. In this PR, it includes the variadic buffers to the result. A test case has also being added. ### Are these changes tested? Yes ### Are there any user-facing changes? No * GitHub Issue: #40930 Authored-by: Vibhatha Lakmal Abeykoon Signed-off-by: David Li --- .../apache/arrow/vector/BaseVariableWidthViewVector.java | 15 +++ .../java/org/apache/arrow/vector/TestVectorReset.java | 12 2 files changed, 19 insertions(+), 8 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java index ec700a0dc2..620b998994 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java @@ -703,13 +703,6 @@ public abstract class BaseVariableWidthViewVector extends BaseValueVector implem * impact the reference counts for this buffer, so it only should be used for in-context * access. Also note that this buffer changes regularly, thus * external classes shouldn't hold a reference to it (unless they change it). - * - * Note: This method only returns validityBuffer and valueBuffer. - * But it doesn't return the data buffers. - * - * TODO: Implement a strategy to retrieve the data buffers. - * https://github.com/apache/arrow/issues/40930;>data buffer retrieval. - * * @param clear Whether to clear vector before returning, the buffers will still be refcounted * but the returned array will be the only reference to them * @return The underlying {@link ArrowBuf buffers} that is used by this @@ -722,9 +715,15 @@ public abstract class BaseVariableWidthViewVector extends BaseValueVector implem if (getBufferSize() == 0) { buffers = new ArrowBuf[0]; } else { - buffers = new ArrowBuf[2]; + final int dataBufferSize = dataBuffers.size(); + // validity and view buffers + final int fixedBufferSize = 2; + buffers = new ArrowBuf[fixedBufferSize + dataBufferSize]; buffers[0] = validityBuffer; buffers[1] = viewBuffer; + for (int i = fixedBufferSize; i < fixedBufferSize + dataBufferSize; i++) { +buffers[i] = dataBuffers.get(i - fixedBufferSize); + } } if (clear) { for (final ArrowBuf buffer : buffers) { diff --git a/java/vector/src/test/java/org/apache/arrow/vector/TestVectorReset.java b/java/vector/src/test/java/org/apache/arrow/vector/TestVectorReset.java index 71009a3337..19700e0216 100644 --- a/java/vector/src/test/java/org/apache/arrow/vector/TestVectorReset.java +++ b/java/vector/src/test/java/org/apache/arrow/vector/TestVectorReset.java @@ -93,6 +93,18 @@ public class TestVectorReset { } } + @Test + public void testVariableViewTypeReset() { +try (final ViewVarCharVector vector = new ViewVarCharVector("ViewVarChar", allocator)) { + vector.allocateNewSafe(); + vector.set(0, "a".getBytes(StandardCharsets.UTF_8)); + vector.setLastSet(0); + vector.setValueCount(1); + resetVectorAndVerify(vector, vector.getBuffers(false)); + assertEquals(-1, vector.getLastSet()); +} + } + @Test public void testLargeVariableTypeReset() { try (final LargeVarCharVector vector = new LargeVarCharVector("LargeVarChar", allocator)) {
(arrow-adbc) branch main updated: feat(rust): add the driver manager (#1803)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 42a6e8113 feat(rust): add the driver manager (#1803) 42a6e8113 is described below commit 42a6e811379719e83483202e1b8f7c4d28eb550c Author: Alexandre Crayssac AuthorDate: Fri May 24 03:51:55 2024 +0200 feat(rust): add the driver manager (#1803) Hey! Here is the penultimate PR containing the driver manager for Rust. The last PR will contain all the integration tests. --- .github/workflows/native-unix.yml|3 +- .github/workflows/native-windows.yml |3 +- .github/workflows/rust.yml | 40 +- rust/core/Cargo.toml |6 +- rust/core/src/driver_exporter.rs |2 +- rust/core/src/driver_manager.rs | 1233 ++ rust/core/src/error.rs | 12 - rust/core/src/ffi/types.rs |5 +- rust/core/src/lib.rs | 12 +- rust/drivers/dummy/src/lib.rs| 10 +- 10 files changed, 1282 insertions(+), 44 deletions(-) diff --git a/.github/workflows/native-unix.yml b/.github/workflows/native-unix.yml index c3f54cf54..144ea9cfc 100644 --- a/.github/workflows/native-unix.yml +++ b/.github/workflows/native-unix.yml @@ -42,9 +42,10 @@ on: - "python/**" - "ruby/**" - ".github/workflows/native-unix.yml" + workflow_call: concurrency: - group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }} + group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }}-unix cancel-in-progress: true permissions: diff --git a/.github/workflows/native-windows.yml b/.github/workflows/native-windows.yml index 5f0964d02..e78403d03 100644 --- a/.github/workflows/native-windows.yml +++ b/.github/workflows/native-windows.yml @@ -40,9 +40,10 @@ on: - "python/**" - "ruby/**" - ".github/workflows/native-windows.yml" + workflow_call: concurrency: - group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }} + group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }}-windows cancel-in-progress: true permissions: diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 793db58a8..8f2ff7ddf 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -30,7 +30,7 @@ on: - ".github/workflows/rust.yml" concurrency: - group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }} + group: ${{ github.repository }}-${{ github.ref }}-${{ github.workflow }}-rust cancel-in-progress: true permissions: @@ -42,11 +42,16 @@ defaults: working-directory: rust jobs: + native-unix: +uses: ./.github/workflows/native-unix.yml + rust: + needs: [native-unix] strategy: matrix: -# TODO(lidavidm): add back macos-latest once it has Cargo installed -os: [windows-latest, macos-13, ubuntu-latest] +# TODO(alexandreyc): add `windows-latest` +# See: https://github.com/apache/arrow-adbc/pull/1803#issuecomment-2117669300 +os: [macos-13, macos-latest, ubuntu-latest] name: "Rust ${{ matrix.os }}" runs-on: ${{ matrix.os }} env: @@ -60,21 +65,24 @@ jobs: - name: Use stable Rust id: rust run: | - rustup update stable + rustup toolchain install stable --no-self-update rustup default stable - VERSION=$(rustup show active-toolchain -v | tail -1 | tr -d "[:space:]") - echo "version=$VERSION" >> "$GITHUB_OUTPUT" - - uses: actions/cache@v4 + - uses: actions/download-artifact@v4 with: - path: | -~/.cargo/bin/ -~/.cargo/registry/index/ -~/.cargo/registry/cache/ -~/.cargo/git/db/ -rust/target/ - key: ${{ runner.os }}-cargo-${{ steps.rust.outputs.version }}-${{ hashFiles('**/Cargo.lock') }} - restore-keys: | -${{ runner.os }}-cargo-${{ steps.rust.outputs.version }}- + name: driver-manager-${{ matrix.os }} + path: ${{ github.workspace }}/build + - name: Set dynamic linker path +if: matrix.os == 'ubuntu-latest' +run: | + echo "LD_LIBRARY_PATH=${{ github.workspace }}/build/lib:$LD_LIBRARY_PATH" >> "$GITHUB_ENV" + - name: Set dynamic linker path +if: matrix.os == 'macos-latest' +run: | + echo "DYLD_LIBRARY_PATH=/opt/homebrew/opt/sqlite/lib:${{ github.workspace }}/build/lib:$DYLD_LIBRARY_PATH" >> "$GITHUB_ENV" + - name: Set dynamic linker path +if: matrix.os == 'macos-13' +run: | +
(arrow) branch main updated: GH-41789: [Java] Clean up immutables and checkerframework dependencies (#41790)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 417a86b9ff GH-41789: [Java] Clean up immutables and checkerframework dependencies (#41790) 417a86b9ff is described below commit 417a86b9ff11493dc35b95f4d8599ea269b1439c Author: Laurent Goujon AuthorDate: Thu May 23 18:36:57 2024 -0700 GH-41789: [Java] Clean up immutables and checkerframework dependencies (#41790) ### Rationale for this change As annotation processing is directly configured in `maven-compiler-plugin`, project dependencies should not include annotation processors in their dependencies, and annotations should be limited to `provided` scope as much as possible. ### What changes are included in this PR? Clean up immutables and checkerframework dependencies to address the issue above: * switch from `org.immutables:value` to `org.immutables:value-annotations` * update `org.immutables` dependencies from 2.10.0 to 2.10.1 * change `org.checkerframework:checker-qual` default scope from `compile` to `provided` * add `org.immutables:value` and `org.checkerframework:checker` to the list of banned dependencies ### Are these changes tested? CI only ### Are there any user-facing changes? No * GitHub Issue: #41789 Authored-by: Laurent Goujon Signed-off-by: David Li --- java/adapter/avro/pom.xml | 2 +- java/adapter/jdbc/pom.xml | 2 +- java/adapter/orc/pom.xml | 2 +- java/algorithm/pom.xml | 2 +- java/c/pom.xml | 2 +- java/compression/pom.xml | 2 +- java/dataset/pom.xml | 2 +- java/flight/flight-core/pom.xml| 2 +- java/flight/flight-sql/pom.xml | 2 +- java/gandiva/pom.xml | 2 +- java/memory/memory-core/pom.xml| 2 +- java/memory/memory-core/src/main/java/module-info.java | 2 -- java/memory/memory-netty/pom.xml | 2 +- java/memory/memory-unsafe/pom.xml | 2 +- java/pom.xml | 11 +++ java/tools/pom.xml | 2 +- java/vector/pom.xml| 2 +- 17 files changed, 22 insertions(+), 21 deletions(-) diff --git a/java/adapter/avro/pom.xml b/java/adapter/avro/pom.xml index 9ddc150253..0046fcac62 100644 --- a/java/adapter/avro/pom.xml +++ b/java/adapter/avro/pom.xml @@ -47,7 +47,7 @@ org.immutables - value + value-annotations diff --git a/java/adapter/jdbc/pom.xml b/java/adapter/jdbc/pom.xml index 5f72729bb7..17681538ac 100644 --- a/java/adapter/jdbc/pom.xml +++ b/java/adapter/jdbc/pom.xml @@ -48,7 +48,7 @@ org.immutables - value + value-annotations diff --git a/java/adapter/orc/pom.xml b/java/adapter/orc/pom.xml index f6aadca6de..ca817510bf 100644 --- a/java/adapter/orc/pom.xml +++ b/java/adapter/orc/pom.xml @@ -46,7 +46,7 @@ org.immutables - value + value-annotations org.apache.orc diff --git a/java/algorithm/pom.xml b/java/algorithm/pom.xml index 92fa5c8553..0854da48b7 100644 --- a/java/algorithm/pom.xml +++ b/java/algorithm/pom.xml @@ -44,7 +44,7 @@ org.immutables - value + value-annotations diff --git a/java/c/pom.xml b/java/c/pom.xml index 1095e99bbd..bfb233315a 100644 --- a/java/c/pom.xml +++ b/java/c/pom.xml @@ -50,7 +50,7 @@ org.immutables - value + value-annotations org.apache.arrow diff --git a/java/compression/pom.xml b/java/compression/pom.xml index ff1c7e1690..26467dbaf2 100644 --- a/java/compression/pom.xml +++ b/java/compression/pom.xml @@ -37,7 +37,7 @@ org.immutables - value + value-annotations org.apache.commons diff --git a/java/dataset/pom.xml b/java/dataset/pom.xml index dd0c76523d..3dea16204a 100644 --- a/java/dataset/pom.xml +++ b/java/dataset/pom.xml @@ -46,7 +46,7 @@ org.immutables - value + value-annotations org.apache.arrow diff --git a/java/flight/flight-core/pom.xml b/java/flight/flight-core/pom.xml index a7ea80d68f..b565572b38 100644 --- a/java/flight/flight-core/pom.xml +++ b/java/flight/flight-core/pom.xml @@ -119,7 +119,7 @@ org.immutables - value + value-annotations diff --git a/java/flight/flight-sql/pom.xml b/java/flight/flight-sql/pom.xml index d5366ae988
(arrow) branch dependabot/maven/java/com.google.api.grpc-proto-google-common-protos-2.39.1 deleted (was 68627019e7)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/com.google.api.grpc-proto-google-common-protos-2.39.1 in repository https://gitbox.apache.org/repos/asf/arrow.git was 68627019e7 MINOR: [Java] Bump com.google.api.grpc:proto-google-common-protos The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow) branch main updated: MINOR: [Java] Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.39.1 in /java (#41740)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 0a79448ba3 MINOR: [Java] Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.39.1 in /java (#41740) 0a79448ba3 is described below commit 0a79448ba39496ec4876cf2f1f0d083bf12152c5 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Fri May 24 10:36:22 2024 +0900 MINOR: [Java] Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.39.1 in /java (#41740) Bumps [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) from 2.37.1 to 2.39.1. Release notes Sourced from https://github.com/googleapis/sdk-platform-java/releases;>com.google.api.grpc:proto-google-common-protos's releases. v2.39.0 https://github.com/googleapis/sdk-platform-java/compare/v2.38.1...v2.39.0;>2.39.0 (2024-04-18) Features add libraries_bom_version to generation configuration (https://redirect.github.com/googleapis/sdk-platform-java/issues/2639;>#2639) (https://github.com/googleapis/sdk-platform-java/commit/56c7ca51d4ca99629251e0aa9cac316eb349c974;>56c7ca5) Add ChannelPoolSettings Getter for gRPC's ChannelProvider (https://redirect.github.com/googleapis/sdk-platform-java/issues/2612;>#2612) (https://github.com/googleapis/sdk-platform-java/commit/d0c5191525d5d157812cae9551fc15bb70caab7e;>d0c5191) add config change (https://redirect.github.com/googleapis/sdk-platform-java/issues/2604;>#2604) (https://github.com/googleapis/sdk-platform-java/commit/8312706db6352fca05a5858b513e88f162e5de0b;>8312706) add entry point (https://redirect.github.com/googleapis/sdk-platform-java/issues/2616;>#2616) (https://github.com/googleapis/sdk-platform-java/commit/b19fa3373d3fa17531d627d4581d154374a231ff;>b19fa33) add generation config comparator (https://redirect.github.com/googleapis/sdk-platform-java/issues/2587;>#2587) (https://github.com/googleapis/sdk-platform-java/commit/a94c2f0e8a99f0ddf17106cbc8117cefe6b0e127;>a94c2f0) Add JavadocJar Task to build.gradle for self service libraries (https://redirect.github.com/googleapis/sdk-platform-java/issues/2593;>#2593) (https://github.com/googleapis/sdk-platform-java/commit/993f5acf91de3a9d2c7f9fdb10ea47387b240554;>993f5ac) Client/StubSettings' getEndpoint() returns the resolved endpoint (https://redirect.github.com/googleapis/sdk-platform-java/issues/2440;>#2440) (https://github.com/googleapis/sdk-platform-java/commit/4942bc17e9c7261242ba3d03d85cd8b131ca2e5e;>4942bc1) generate selected libraries (https://redirect.github.com/googleapis/sdk-platform-java/issues/2598;>#2598) (https://github.com/googleapis/sdk-platform-java/commit/739ddbbbc662f43e46b9f420bbae685de9b3dbc6;>739ddbb) Validate the Universe Domain inside Java-Core (https://redirect.github.com/googleapis/sdk-platform-java/issues/2592;>#2592) (https://github.com/googleapis/sdk-platform-java/commit/35d789fdf488feff41d4bdaa5992013450954870;>35d789f) Bug Fixes add main to generate_repo.py (https://redirect.github.com/googleapis/sdk-platform-java/issues/2607;>#2607) (https://github.com/googleapis/sdk-platform-java/commit/fedeb3237b6d25245f4d770ec105bd538dfee405;>fedeb32) correct deep-remove and deep-preserve regexes (https://redirect.github.com/googleapis/sdk-platform-java/issues/2572;>#2572) (https://github.com/googleapis/sdk-platform-java/commit/4c7fd880a90d08448db366bde40d0d385b45f75e;>4c7fd88) first attempt should use the min of RPC timeout and total timeout (https://redirect.github.com/googleapis/sdk-platform-java/issues/2641;>#2641) (https://github.com/googleapis/sdk-platform-java/commit/034923277750171418d74060e225ffd68ef38bf2;>0349232) remove duplicated calls to AutoValue builders (https://redirect.github.com/googleapis/sdk-platform-java/issues/2636;>#2636) (https://github.com/googleapis/sdk-platform-java/commit/53a3727e4a9548bf0417d4ee6ed3e9ab571642ed;>53a3727) remove unnecessary slf4j and AbstractGoogleClientRequest native image configs (https://github.com/googleapis/sdk-platform-java/commit/0cb7d0e0c03f2dd2853a86d67c9529a50ed8238f;>0cb7d0e) remove unnecessary slf4j and AbstractGoogleClientRequest native image configs (https://redirect.github.com/googleapis/sdk-platform-java/issues/2628;>#2628) (https://github.com/googleapis/sdk-platform-java/commit/0cb7d0e0c03f2dd2853a86d67c9529a50ed8238f;>0cb7d0e) Dependencies update arrow.version to v15.0.2 (https://redirect.github.com/googleapis/sdk-platform-java/issues/2589;>#2589) (https://github.com/googleapis/sdk-platform-java/commit/777acf3df4b5379cd8a8cf52b844203dd514ee24;>77
(arrow) branch main updated: GH-41787: Update fmpp-maven-plugin output directory (#41788)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 420c01ab98 GH-41787: Update fmpp-maven-plugin output directory (#41788) 420c01ab98 is described below commit 420c01ab98295c7ee2f3ef640da911e4e8276a1c Author: Laurent Goujon AuthorDate: Wed May 22 18:55:38 2024 -0700 GH-41787: Update fmpp-maven-plugin output directory (#41788) ### Rationale for this change Per convention fmpp-maven-plugin should not directly generate files under target/generated-sources but into a subdirectory. ### What changes are included in this PR? Changing config to output files under `target/generated-sources/fmpp` ### Are these changes tested? CI ### Are there any user-facing changes? No * GitHub Issue: #41787 Authored-by: Laurent Goujon Signed-off-by: David Li --- java/vector/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java/vector/pom.xml b/java/vector/pom.xml index 07af93a499..a315bbc03a 100644 --- a/java/vector/pom.xml +++ b/java/vector/pom.xml @@ -199,7 +199,7 @@ generate-sources src/main/codegen/config.fmpp - ${project.build.directory}/generated-sources + ${project.build.directory}/generated-sources/fmpp ${project.build.directory}/codegen/templates
(arrow-site) branch main updated: [Website] Add ADBC 12 release post (#523)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-site.git The following commit(s) were added to refs/heads/main by this push: new 2a0acf897b6 [Website] Add ADBC 12 release post (#523) 2a0acf897b6 is described below commit 2a0acf897b610a64e45a9bb54e3b1991868f4837 Author: David Li AuthorDate: Thu May 23 09:24:32 2024 +0900 [Website] Add ADBC 12 release post (#523) --- _posts/2024-05-21-adbc-12-release.md | 100 +++ 1 file changed, 100 insertions(+) diff --git a/_posts/2024-05-21-adbc-12-release.md b/_posts/2024-05-21-adbc-12-release.md new file mode 100644 index 000..b02785be09f --- /dev/null +++ b/_posts/2024-05-21-adbc-12-release.md @@ -0,0 +1,100 @@ +--- +layout: post +title: "Apache Arrow ADBC 12 (Libraries) Release" +date: "2024-05-21 00:00:00" +author: pmc +categories: [release] +--- + + +The Apache Arrow team is pleased to announce the 12th release of +the Apache Arrow ADBC libraries. This covers includes [**56 +resolved issues**][1] from [**13 distinct contributors**][2]. + +This is a release of the **libraries**, which are at version 12. +The **API specification** is versioned separately and is at +version 1.1.0. + +The subcomponents are versioned independently: + +- C/C++/GLib/Go/Python/Ruby: 1.0.0 +- C#: 0.12.0 +- Java: 0.12.0 +- R: 0.12.0 +- Rust: 0.12.0 + +The release notes below are not exhaustive and only expose selected +highlights of the release. Many other bugfixes and improvements have +been made: we refer you to the [complete changelog][3]. + +## Release Highlights + +There is a [known issue](https://github.com/apache/arrow-adbc/issues/1841) +with multiple drivers in a single process due to using languages with runtimes +as the basis for many drivers. + +Option strings in C# are now case-sensitive. In general, the C# bindings are +rapidly progressing. A driver that wraps the Hive Thrift API was added. + +The Flight SQL driver now supports the new "stateless" prepared statement +proposal. + +Rust libraries were not released to Cargo, but the implementation has made +rapid progress. + +The Snowflake driver now supports bind parameters. Also, some queries which +only return JSON data (e.g. `SHOW TABLES`) are now supported. Quoting of +names in bulk ingestion was fixed to conform to Snowflake SQL syntax. + +## Contributors + +``` +$ git shortlog --perl-regexp --author='^((?!dependabot\[bot\]).*)$' -sn apache-arrow-adbc-0.11.0..apache-arrow-adbc-12 +25 David Li +19 Curt Hagenlocher + 5 Matt Topol + 5 Sutou Kouhei + 4 Dewey Dunnington + 3 Alexandre Crayssac + 3 Matthijs Brobbel + 2 Bruce Irschick + 2 Cocoa + 2 davidhcoe + 1 Bryce Mecum + 1 Hyunseok Seo + 1 Joel Lubinitsky +``` + +## Roadmap + +A Google BigQuery driver is being developed in Go. We anticipate C# will +reach stability soon and Rust should start seeing releases as well. + +## Getting Involved + +We welcome questions and contributions from all interested. Issues +can be filed on [GitHub][4], and questions can be directed to GitHub +or the [Arrow mailing lists][5]. + +[1]: https://github.com/apache/arrow-adbc/milestone/16 +[2]: #contributors +[3]: https://github.com/apache/arrow-adbc/blob/apache-arrow-adbc-12/CHANGELOG.md +[4]: https://github.com/apache/arrow-adbc/issues +[5]: {% link community.md %}
(arrow-adbc) branch main updated: chore: bump to release 13, fix other post-release scripts (#1876)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new a40cf8840 chore: bump to release 13, fix other post-release scripts (#1876) a40cf8840 is described below commit a40cf88408d6cb776cedeaa4d1d0945675c156cc Author: David Li AuthorDate: Wed May 22 10:40:57 2024 +0900 chore: bump to release 13, fix other post-release scripts (#1876) --- CHANGELOG.md | 94 ++ c/cmake_modules/AdbcVersion.cmake | 2 +- ci/conda/meta.yaml | 2 +- ci/linux-packages/debian/control | 22 ++--- ...install => libadbc-driver-flightsql101.install} | 0 ...0.install => libadbc-driver-manager101.install} | 0 ...nstall => libadbc-driver-postgresql101.install} | 0 ...install => libadbc-driver-snowflake101.install} | 0 ...00.install => libadbc-driver-sqlite101.install} | 0 csharp/Directory.Build.props | 2 +- dev/release/post-09-bump-versions.sh | 7 +- dev/release/utils-prepare.sh | 14 ++-- dev/release/versions.env | 18 +++-- docs/source/conf.py| 2 +- glib/meson.build | 2 +- java/core/pom.xml | 2 +- java/driver-manager/pom.xml| 2 +- java/driver/flight-sql-validation/pom.xml | 2 +- java/driver/flight-sql/pom.xml | 2 +- java/driver/jdbc-validation-derby/pom.xml | 2 +- java/driver/jdbc-validation-mssqlserver/pom.xml| 2 +- java/driver/jdbc-validation-postgresql/pom.xml | 2 +- java/driver/jdbc/pom.xml | 2 +- java/driver/validation/pom.xml | 2 +- java/pom.xml | 4 +- java/sql/pom.xml | 2 +- .../adbc_driver_flightsql/_static_version.py | 2 +- .../adbc_driver_manager/_static_version.py | 2 +- .../adbc_driver_postgresql/_static_version.py | 2 +- .../adbc_driver_snowflake/_static_version.py | 2 +- .../adbc_driver_sqlite/_static_version.py | 2 +- r/adbcdrivermanager/DESCRIPTION| 2 +- r/adbcflightsql/DESCRIPTION| 2 +- r/adbcpostgresql/DESCRIPTION | 2 +- r/adbcsnowflake/DESCRIPTION| 2 +- r/adbcsqlite/DESCRIPTION | 2 +- ruby/lib/adbc/version.rb | 2 +- rust/Cargo.toml| 2 +- 38 files changed, 153 insertions(+), 60 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ee1ddfcd..660a192bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -511,3 +511,97 @@ - **c/driver_manager**: differentiate errors from driver manager (#1662) - **c/driver/sqlite**: port to driver base (#1603) - **go/adbc/driver**: driverbase implementation for connection (#1590) + +## ADBC Libraries 12 (2024-05-09) + +### Fix + +- **csharp/src/Apache.Arrow.Adbc/C**: finalizer threw exception (#1842) +- **dev/release**: handle versioning scheme in binary verification (#1834) +- **csharp**: Change option translation to be case-sensitive (#1820) +- **csharp/src/Apache.Arrow.Adbc/C**: imported errors don't return a native error or SQL state (#1815) +- **csharp/src/Apache.Arrow.Adbc**: imported statements and databases don't allow options to be set (#1816) +- **csharp/src/Apache.Arrow.Adbc/C**: correctly handle null driver entries for imported drivers (#1812) +- **go/adbc/driver/snowflake**: handle empty result sets (#1805) +- **csharp**: an assortment of small fixes not worth individual pull requests (#1807) +- **go/adbc/driver/snowflake**: workaround snowflake metadata-only limitations (#1790) +- **csharp/src/Apache.Arrow.Adbc**: correct StandardSchemas.ColumnSchema data types (#1731) +- **csharp**: imported drivers have the potential for a lot of memory leaks (#1776) +- **go/adbc/driver/flightsql**: should use `ctx.Err().Error()` (#1769) +- **go/adbc/driver/snowflake**: handle quotes properly (#1738) +- **go/adbc/driver/snowflake**: comment format (#1768) +- **csharp/src/Apache.Arrow.Adbc**: Fix marshaling in three functions where it was broken (#1758) +- **csharp/src/Apache.Arrow.Adbc**: Add support to the C Exporter for converting exceptions into AdbcErrors (#1752) +- **format**: correct duplicated statistics names (#1736) +- **dev/release**: correct C# version bump regex (#1733) +- **r**: Ensure CXX_STD is set everywhere (#1706) +- **csharp**: Resolve memory leaks described by #1690 (#1695) + +### Feat + +- **go/adbc/driver/snowflake**: support parameter binding (#1808) +- **csharp**: imported
(arrow-adbc) branch main updated: chore(dev/bench): add a simple python benchmark script for ADBC (#1879)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new d4933e410 chore(dev/bench): add a simple python benchmark script for ADBC (#1879) d4933e410 is described below commit d4933e4104e08cf1154b6870af442072ffa7b46e Author: Matt Topol AuthorDate: Tue May 21 20:33:30 2024 -0400 chore(dev/bench): add a simple python benchmark script for ADBC (#1879) Just a starting point to hopefully continue building up some benchmarking suite stuff. --- .pre-commit-config.yaml | 4 +- dev/bench/README.md | 30 dev/bench/run_bench.py | 177 3 files changed, 209 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 2b726f1b4..fafc20df4 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -40,7 +40,7 @@ repos: - id: trailing-whitespace exclude: "^r/.*?/_snaps/.*?.md$" - repo: https://github.com/pre-commit/mirrors-clang-format -rev: "v18.1.4" +rev: "v18.1.5" hooks: - id: clang-format types_or: [c, c++] @@ -59,7 +59,7 @@ repos: - "--linelength=90" - "--verbose=2" - repo: https://github.com/golangci/golangci-lint -rev: v1.57.2 +rev: v1.58.2 hooks: - id: golangci-lint entry: bash -c 'cd go/adbc && golangci-lint run --fix --timeout 5m' diff --git a/dev/bench/README.md b/dev/bench/README.md new file mode 100644 index 0..1c332885d --- /dev/null +++ b/dev/bench/README.md @@ -0,0 +1,30 @@ + + +# Simple Python Benchmark script + +Connection parameters need to be filled in before the script can be run. The intent +is for this to be a simple enough script to provide iterations on running a `SELECT` +query solely for testing data transfer and memory usage rates for simple queries. + +The initial sample here is designed for testing against Snowflake, and so contains +functions for testing the ADBC Snowflake driver, the [snowflake-python-connector](https://pypi.org/project/snowflake-connector-python/), and using ODBC via pyodbc. + +If `matplotlib` is installed, it will also draw the timing and memory usage up as +charts which can be saved. diff --git a/dev/bench/run_bench.py b/dev/bench/run_bench.py new file mode 100644 index 0..0ce2fbcf3 --- /dev/null +++ b/dev/bench/run_bench.py @@ -0,0 +1,177 @@ +# #!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import argparse +import time + +import psutil + +import adbc_driver_snowflake.dbapi + +# import pyodbc +# import snowflake.connector + + +process = psutil.Process() + +SAMPLE_RATE = 10 # record data every SAMPLE_RATE execution + +can_draw = True +try: +import matplotlib.pyplot as plt +except ImportError: +print("graphs cannot be drawn as matplotlib is not installed") +can_draw = False + + +def task_execution_decorator(func, perf_file, memory_file): +count = 0 + +def wrapper(*args, **kwargs): +start = time.time() +func(*args, **kwargs) +memory_usage = ( +process.memory_info().rss / 1024 / 1024 +) # rss is in bytes, we convert to MB +period = time.time() - start +nonlocal count +if count % SAMPLE_RATE == 0: +perf_file.write(str(period) + "\n") +print(f"execution {count}") +print(f"memory usage: {memory_usage} MB") +print(f"execution time: {period} s") +memory_file.write(str(memory_usage) + "\n") +count += 1 + +return wrapper + + +def task_fetch_arrow_batches(cursor, table_name, row_count_limit=5): +ret = cursor.execute( +f"select * from {table_name} limit {row_count_limit}" +).fetch_arrow_batches() # interface for snowflake-python-connector +for _ in ret: +pass + + +def task_fetch_record_batch(cursor, table_name, row_count_limit=5):
(arrow) branch main updated: GH-41571: [Java] Revert GH-41307 (#41309) (#41628)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new e3cd0ae3ea GH-41571: [Java] Revert GH-41307 (#41309) (#41628) e3cd0ae3ea is described below commit e3cd0ae3ea648ce5b78a2d9c9e21c54d772cb385 Author: David Li AuthorDate: Tue May 21 19:11:52 2024 +0900 GH-41571: [Java] Revert GH-41307 (#41309) (#41628) ### Rationale for this change The commit in question caused a lot of CI issues ### Are these changes tested? N/A ### Are there any user-facing changes? N/A * GitHub Issue: #41571 Authored-by: David Li Signed-off-by: David Li --- java/adapter/avro/pom.xml | 9 + java/adapter/jdbc/pom.xml | 7 + java/adapter/orc/pom.xml | 17 -- java/bom/pom.xml | 21 +-- java/c/pom.xml | 1 + java/dataset/pom.xml | 2 +- java/flight/flight-core/pom.xml| 27 ++- java/flight/flight-integration-tests/pom.xml | 2 + java/flight/flight-sql-jdbc-core/pom.xml | 10 + java/flight/flight-sql-jdbc-driver/pom.xml | 1 + java/flight/flight-sql/pom.xml | 5 + java/format/pom.xml| 2 + java/gandiva/pom.xml | 19 +- .../module-info-compiler-maven-plugin/pom.xml | 28 ++- java/maven/pom.xml | 75 +--- java/memory/memory-core/pom.xml| 22 ++- java/performance/pom.xml | 40 java/pom.xml | 207 - java/tools/pom.xml | 22 +-- java/vector/pom.xml| 91 - 20 files changed, 432 insertions(+), 176 deletions(-) diff --git a/java/adapter/avro/pom.xml b/java/adapter/avro/pom.xml index 645e8c4ff2..9ddc150253 100644 --- a/java/adapter/avro/pom.xml +++ b/java/adapter/avro/pom.xml @@ -25,27 +25,36 @@ http://maven.apache.org + + org.apache.arrow arrow-memory-core + + org.apache.arrow arrow-memory-netty runtime + + org.apache.arrow arrow-vector + org.immutables value + org.apache.avro avro ${dep.avro.version} + diff --git a/java/adapter/jdbc/pom.xml b/java/adapter/jdbc/pom.xml index 33360c64b1..5f72729bb7 100644 --- a/java/adapter/jdbc/pom.xml +++ b/java/adapter/jdbc/pom.xml @@ -26,17 +26,20 @@ + org.apache.arrow arrow-memory-core + org.apache.arrow arrow-memory-netty runtime + org.apache.arrow arrow-vector @@ -48,6 +51,7 @@ value + com.h2database h2 @@ -90,6 +94,9 @@ jdk11+ [11,] + + !m2e.version + diff --git a/java/adapter/orc/pom.xml b/java/adapter/orc/pom.xml index ec6f73a3e9..f6aadca6de 100644 --- a/java/adapter/orc/pom.xml +++ b/java/adapter/orc/pom.xml @@ -134,22 +134,5 @@ - - -org.apache.maven.plugins -maven-dependency-plugin - - -analyze - - - - org.apache.arrow:arrow-format - - - - - - diff --git a/java/bom/pom.xml b/java/bom/pom.xml index 0af50c6380..12b9950ad8 100644 --- a/java/bom/pom.xml +++ b/java/bom/pom.xml @@ -15,7 +15,7 @@ org.apache apache -31 +18 org.apache.arrow @@ -27,19 +27,6 @@ - -1.8 -1.8 -3.11.0 -3.2.5 -0.16.1 -3.7.1 -3.12.1 -3.6.1 -3.2.0 -3.2.2 -3.6.3 - 3.5.0 @@ -151,9 +138,11 @@ ${project.version} + + @@ -167,10 +156,12 @@ org.apache.maven.plugins maven-project-info-reports-plugin +3.5.0 org.apache.maven.plugins maven-site-plugin +3.12.1 com.diffplug.spotless @@ -197,10 +188,12 @@ org.apache.maven.plugins maven-project-info-reports-plugin +3.5.0 org.apache.maven.plugins maven-site-plugin +3.12.1 diff --git a/java/c/pom.xml b/java/c/pom.xml index 43a62a8303..1095e99bbd 100644 --- a/java/c/pom.xml +++ b/java/c/pom.xml @@ -83,4 +83,5 @@ + diff --git a/java/dataset/pom.xml b
svn commit: r69308 - /release/arrow/apache-arrow-adbc-0.11.0/
Author: lidavidm Date: Tue May 21 00:17:32 2024 New Revision: 69308 Log: Remove old Apache Arrow ADBC release: apache-arrow-adbc-0.11.0/ Removed: release/arrow/apache-arrow-adbc-0.11.0/
svn commit: r69307 - /dev/arrow/apache-arrow-adbc-12-rc4/
Author: lidavidm Date: Tue May 21 00:17:29 2024 New Revision: 69307 Log: Remove old Apache Arrow ADBC RC: apache-arrow-adbc-12-rc4/ Removed: dev/arrow/apache-arrow-adbc-12-rc4/
svn commit: r69306 - /dev/arrow/apache-arrow-adbc-12-rc3/
Author: lidavidm Date: Tue May 21 00:17:27 2024 New Revision: 69306 Log: Remove old Apache Arrow ADBC RC: apache-arrow-adbc-12-rc3/ Removed: dev/arrow/apache-arrow-adbc-12-rc3/
svn commit: r69305 - /dev/arrow/apache-arrow-adbc-12-rc0/
Author: lidavidm Date: Tue May 21 00:17:25 2024 New Revision: 69305 Log: Remove old Apache Arrow ADBC RC: apache-arrow-adbc-12-rc0/ Removed: dev/arrow/apache-arrow-adbc-12-rc0/
(arrow-adbc) annotated tag go/adbc/v1.0.0 updated (50cb9de62 -> cb1fd9832)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag go/adbc/v1.0.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag go/adbc/v1.0.0 was modified! *** from 50cb9de62 (commit) to cb1fd9832 (tag) tagging 50cb9de621c4d72f4aefd18237cb4b73b82f4a0e (commit) replaces apache-arrow-adbc-12-rc3 by David Li on Tue May 21 08:57:17 2024 +0900 - Log - ADBC Libraries 12 --- No new revisions were added by this update. Summary of changes:
(arrow-adbc) annotated tag apache-arrow-adbc-12 updated (50cb9de62 -> cb1fd9832)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12 was modified! *** from 50cb9de62 (commit) to cb1fd9832 (tag) tagging 50cb9de621c4d72f4aefd18237cb4b73b82f4a0e (commit) replaces apache-arrow-adbc-12-rc3 by David Li on Tue May 21 08:57:17 2024 +0900 - Log - ADBC Libraries 12 --- No new revisions were added by this update. Summary of changes:
svn commit: r69304 - /release/arrow/apache-arrow-adbc-12/
Author: lidavidm Date: Mon May 20 23:57:17 2024 New Revision: 69304 Log: Apache Arrow ADBC 12 Added: release/arrow/apache-arrow-adbc-12/ - copied from r69303, dev/arrow/apache-arrow-adbc-12-rc4/
(arrow) branch main updated: GH-41717: [Java][Vector] fix issue with ByteBuffer rewind in MessageSerializer (#41718)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 1cd28729a3 GH-41717: [Java][Vector] fix issue with ByteBuffer rewind in MessageSerializer (#41718) 1cd28729a3 is described below commit 1cd28729a34a0e761c7dba2612e7f9ec6f4ea31c Author: PJ Fanning AuthorDate: Mon May 20 23:56:48 2024 +0100 GH-41717: [Java][Vector] fix issue with ByteBuffer rewind in MessageSerializer (#41718) ### Rationale for this change ### What changes are included in this PR? #41717 describes issue and change ### Are these changes tested? CI build ### Are there any user-facing changes? * GitHub Issue: #41717 Authored-by: PJ Fanning Signed-off-by: David Li --- .../java/org/apache/arrow/vector/ipc/message/MessageSerializer.java| 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java b/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java index 9deb42c498..099103cd17 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/ipc/message/MessageSerializer.java @@ -701,7 +701,8 @@ public class MessageSerializer { throw new IOException( "Unexpected end of stream trying to read message."); } -messageBuffer.rewind(); +// see https://github.com/apache/arrow/issues/41717 for reason why we cast to java.nio.Buffer +ByteBuffer rewindBuffer = (ByteBuffer) ((java.nio.Buffer) messageBuffer).rewind(); // Load the message. Message message = Message.getRootAsMessage(messageBuffer);
(arrow) branch main updated (1c15c88b4b -> 0574988e32)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git from 1c15c88b4b GH-40943: [Java] Implement RangeEqualsVisitor for StringView (#41636) add 0574988e32 GH-41287: [Java] ListViewVector Implementation (#41285) No new revisions were added by this update. Summary of changes: .../jdbc/binder/ColumnBinderArrowTypeVisitor.java |5 + .../apache/arrow/c/BufferImportTypeVisitor.java|6 + .../driver/jdbc/utils/AvaticaParameterBinder.java |5 + .../arrow/driver/jdbc/utils/ConvertUtils.java |5 + java/vector/src/main/codegen/data/ArrowTypes.tdd |5 + .../main/codegen/templates/UnionListWriter.java| 24 +- .../src/main/codegen/templates/UnionReader.java|2 +- .../java/org/apache/arrow/vector/BufferLayout.java | 13 +- .../java/org/apache/arrow/vector/TypeLayout.java | 20 +- ...ector.java => BaseRepeatedValueViewVector.java} | 266 ++-- .../{ListVector.java => ListViewVector.java} | 724 - .../vector/complex/impl/PromotableWriter.java | 30 + .../java/org/apache/arrow/vector/types/Types.java | 21 + .../apache/arrow/vector/TestListViewVector.java| 1651 14 files changed, 2243 insertions(+), 534 deletions(-) copy java/vector/src/main/java/org/apache/arrow/vector/complex/{BaseRepeatedValueVector.java => BaseRepeatedValueViewVector.java} (60%) copy java/vector/src/main/java/org/apache/arrow/vector/complex/{ListVector.java => ListViewVector.java} (54%) create mode 100644 java/vector/src/test/java/org/apache/arrow/vector/TestListViewVector.java
(arrow) branch main updated: GH-40943: [Java] Implement RangeEqualsVisitor for StringView (#41636)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 1c15c88b4b GH-40943: [Java] Implement RangeEqualsVisitor for StringView (#41636) 1c15c88b4b is described below commit 1c15c88b4b62b19e7a226cc9e11946af1d9ac343 Author: Vibhatha Lakmal Abeykoon AuthorDate: Thu May 16 06:38:57 2024 +0530 GH-40943: [Java] Implement RangeEqualsVisitor for StringView (#41636) ### Rationale for this change Adding `RangeEqualsVisitor` for StringView as discussed in https://github.com/apache/arrow/issues/40943. ### What changes are included in this PR? Including `RangeEqualsVisitor` visitor method and test cases to validate it. ### Are these changes tested? Yes ### Are there any user-facing changes? No * GitHub Issue: #40943 Authored-by: Vibhatha Abeykoon Signed-off-by: David Li --- .../arrow/vector/BaseVariableWidthViewVector.java | 21 -- .../arrow/vector/compare/RangeEqualsVisitor.java | 85 +- .../vector/compare/TestRangeEqualsVisitor.java | 71 +++--- 3 files changed, 161 insertions(+), 16 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java index 2f80775a48..ec700a0dc2 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/BaseVariableWidthViewVector.java @@ -46,7 +46,7 @@ import org.apache.arrow.vector.util.TransferPair; */ public abstract class BaseVariableWidthViewVector extends BaseValueVector implements VariableWidthFieldVector { // A single element of a view comprises 16 bytes - protected static final int ELEMENT_SIZE = 16; + public static final int ELEMENT_SIZE = 16; public static final int INITIAL_VIEW_VALUE_ALLOCATION = 4096; private static final int INITIAL_BYTE_COUNT = INITIAL_VIEW_VALUE_ALLOCATION * ELEMENT_SIZE; private static final int MAX_BUFFER_SIZE = (int) Math.min(MAX_ALLOCATION_SIZE, Integer.MAX_VALUE); @@ -70,14 +70,14 @@ public abstract class BaseVariableWidthViewVector extends BaseValueVector implem * * */ // 12 byte unsigned int to track inline views - protected static final int INLINE_SIZE = 12; + public static final int INLINE_SIZE = 12; // The first 4 bytes of view are allocated for length - protected static final int LENGTH_WIDTH = 4; + public static final int LENGTH_WIDTH = 4; // The second 4 bytes of view are allocated for prefix width - protected static final int PREFIX_WIDTH = 4; + public static final int PREFIX_WIDTH = 4; // The third 4 bytes of view are allocated for buffer index - protected static final int BUF_INDEX_WIDTH = 4; - protected static final byte[] EMPTY_BYTE_ARRAY = new byte[]{}; + public static final int BUF_INDEX_WIDTH = 4; + public static final byte[] EMPTY_BYTE_ARRAY = new byte[]{}; protected ArrowBuf validityBuffer; // The view buffer is used to store the variable width view elements protected ArrowBuf viewBuffer; @@ -158,6 +158,15 @@ public abstract class BaseVariableWidthViewVector extends BaseValueVector implem return viewBuffer; } + /** + * Get the buffers that store the data for views in the vector. + * + * @return buffer + */ + public List getDataBuffers() { +return dataBuffers; + } + /** * BaseVariableWidthViewVector doesn't support offset buffer. * diff --git a/java/vector/src/main/java/org/apache/arrow/vector/compare/RangeEqualsVisitor.java b/java/vector/src/main/java/org/apache/arrow/vector/compare/RangeEqualsVisitor.java index 56220d270f..28da2a86a5 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/compare/RangeEqualsVisitor.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/compare/RangeEqualsVisitor.java @@ -22,6 +22,7 @@ import static org.apache.arrow.memory.util.LargeMemoryUtil.checkedCastToInt; import java.util.List; import java.util.function.BiFunction; +import org.apache.arrow.memory.ArrowBuf; import org.apache.arrow.memory.util.ByteFunctionHelpers; import org.apache.arrow.util.Preconditions; import org.apache.arrow.vector.BaseFixedWidthVector; @@ -165,7 +166,10 @@ public class RangeEqualsVisitor implements VectorVisitor { @Override public Boolean visit(BaseVariableWidthViewVector left, Range range) { -throw new UnsupportedOperationException("View vectors are not supported."); +if (!validate(left)) { + return false; +} +return compareBaseVariableWidthViewVectors(range); } @Override @@ -450,6 +454,85 @@ public class RangeEqualsVisitor implements VectorVisitor { return true; } + protect
svn commit: r69165 - in /dev/arrow/apache-arrow-adbc-12-rc4: ./ apache-arrow-adbc-12.tar.gz apache-arrow-adbc-12.tar.gz.asc apache-arrow-adbc-12.tar.gz.sha512
Author: lidavidm Date: Wed May 15 03:52:17 2024 New Revision: 69165 Log: Apache Arrow ADBC 12 RC4 Added: dev/arrow/apache-arrow-adbc-12-rc4/ dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz (with props) dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.asc (with props) dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.sha512 Added: dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz -- svn:mime-type = application/gzip Added: dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.asc == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.asc -- svn:mime-type = application/pgp-signature Added: dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.sha512 == --- dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.sha512 (added) +++ dev/arrow/apache-arrow-adbc-12-rc4/apache-arrow-adbc-12.tar.gz.sha512 Wed May 15 03:52:17 2024 @@ -0,0 +1 @@ +eb2e15eeedd4dfe911dde11e5d2a929c4e4b13b2e8ff6fb3a47d21b886ffea113ccafbbc0bdd80738645447c5875b4cd1e68e421d277e36fa7f71d5b99b670a3 apache-arrow-adbc-12.tar.gz
(arrow-adbc) branch maint-12 updated (91804736a -> 50cb9de62)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch maint-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from 91804736a chore: update versions for 12 add 38ebbd820 chore: update CHANGELOG.md for 12 add 50cb9de62 chore: update versions for 12 No new revisions were added by this update. Summary of changes: CHANGELOG.md| 47 + ci/linux-packages/debian/changelog | 6 ci/linux-packages/yum/apache-arrow-adbc.spec.in | 3 ++ 3 files changed, 56 insertions(+)
(arrow-adbc) annotated tag apache-arrow-adbc-12-rc4 updated (50cb9de62 -> 42ca8f51f)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12-rc4 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12-rc4 was modified! *** from 50cb9de62 (commit) to 42ca8f51f (tag) tagging 50cb9de621c4d72f4aefd18237cb4b73b82f4a0e (commit) replaces apache-arrow-adbc-12-rc3 by David Li on Wed May 15 09:36:56 2024 +0900 - Log - ADBC Libraries 12 RC 4 --- No new revisions were added by this update. Summary of changes:
(arrow) branch main updated: GH-40944: [Java] Implement TypeEqualsVisitor for StringView (#41606)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 0e9896dc54 GH-40944: [Java] Implement TypeEqualsVisitor for StringView (#41606) 0e9896dc54 is described below commit 0e9896dc54ca82876171521d206ad0906535572f Author: Tai Le Manh <49281946+tlm...@users.noreply.github.com> AuthorDate: Mon May 13 06:54:20 2024 +0700 GH-40944: [Java] Implement TypeEqualsVisitor for StringView (#41606) ### Rationale for this change Resolves #40944 . ### What changes are included in this PR? ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: #40944 Authored-by: Tai Le Manh Signed-off-by: David Li --- .../arrow/vector/compare/TypeEqualsVisitor.java| 2 +- .../vector/compare/TestTypeEqualsVisitor.java | 41 ++ 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/compare/TypeEqualsVisitor.java b/java/vector/src/main/java/org/apache/arrow/vector/compare/TypeEqualsVisitor.java index 9bbe5c1b89..aaef161a56 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/compare/TypeEqualsVisitor.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/compare/TypeEqualsVisitor.java @@ -88,7 +88,7 @@ public class TypeEqualsVisitor implements VectorVisitor { @Override public Boolean visit(BaseVariableWidthViewVector left, Void value) { -throw new UnsupportedOperationException("View vectors are not supported."); +return compareField(left.getField(), right.getField()); } @Override diff --git a/java/vector/src/test/java/org/apache/arrow/vector/compare/TestTypeEqualsVisitor.java b/java/vector/src/test/java/org/apache/arrow/vector/compare/TestTypeEqualsVisitor.java index 62fa0336ea..736b0f1b1a 100644 --- a/java/vector/src/test/java/org/apache/arrow/vector/compare/TestTypeEqualsVisitor.java +++ b/java/vector/src/test/java/org/apache/arrow/vector/compare/TestTypeEqualsVisitor.java @@ -20,6 +20,7 @@ package org.apache.arrow.vector.compare; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertTrue; +import java.nio.charset.StandardCharsets; import java.util.HashMap; import java.util.Map; @@ -30,6 +31,8 @@ import org.apache.arrow.vector.Float4Vector; import org.apache.arrow.vector.Float8Vector; import org.apache.arrow.vector.IntVector; import org.apache.arrow.vector.VarCharVector; +import org.apache.arrow.vector.ViewVarBinaryVector; +import org.apache.arrow.vector.ViewVarCharVector; import org.apache.arrow.vector.complex.DenseUnionVector; import org.apache.arrow.vector.complex.ListVector; import org.apache.arrow.vector.complex.StructVector; @@ -176,4 +179,42 @@ public class TestTypeEqualsVisitor { assertFalse(typeVisitor.equals(vector1)); } } + + @Test + public void testStringViewTypeEquals() { +try (final ViewVarCharVector varchar1 = new ViewVarCharVector("varchar1", allocator); + final ViewVarCharVector varchar2 = new ViewVarCharVector("varchar2", allocator); + final ViewVarBinaryVector binary = new ViewVarBinaryVector("binary", allocator)) { + final int valueCount = 2; + final byte[] str0 = "apache".getBytes(StandardCharsets.UTF_8); + final byte[] str1 = "arrow".getBytes(StandardCharsets.UTF_8); + + // add elements for varchar1 + varchar1.allocateNew(48, valueCount); + varchar1.set(0, str0); + varchar1.set(1, str1); + varchar1.setValueCount(valueCount); + + // add elements for varchar2 in a difference order + varchar2.allocateNew(48, valueCount); + varchar2.set(0, str1); + varchar2.set(1, str0); + varchar2.setValueCount(valueCount); + + // add elements for binary + binary.allocateNew(48, valueCount); + binary.set(0, str0); + binary.set(1, str1); + binary.setValueCount(valueCount); + + // compare ignore check name + TypeEqualsVisitor visitor = new TypeEqualsVisitor(varchar1, /* check name */ false, /* check meta data */ true); + assertTrue(visitor.equals(varchar2)); + assertFalse(visitor.equals(binary)); + + // if we check names, the types should be different + visitor = new TypeEqualsVisitor(varchar1, /* check name */ true, /* check meta data */ true); + assertFalse(visitor.equals(varchar2)); +} + } }
(arrow) branch main updated: GH-41590: [Java] Improve BaseRepeatedValueVector function on isEmpty and isNull operations (#41601)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new f8d3b10b4b GH-41590: [Java] Improve BaseRepeatedValueVector function on isEmpty and isNull operations (#41601) f8d3b10b4b is described below commit f8d3b10b4b89b47f6e7a594b95c82e2ff161f1a5 Author: Tai Le Manh <49281946+tlm...@users.noreply.github.com> AuthorDate: Fri May 10 12:42:25 2024 +0700 GH-41590: [Java] Improve BaseRepeatedValueVector function on isEmpty and isNull operations (#41601) ### Rationale for this change Resolves #41590 . ### What changes are included in this PR? Make `abstract` on `isNull` and `isEmpty` of `BaseRepeatedValueVector`. ### Are these changes tested? Existing tests pass. ### Are there any user-facing changes? No. * GitHub Issue: #41590 Authored-by: Tai Le Manh Signed-off-by: David Li --- .../apache/arrow/vector/complex/BaseRepeatedValueVector.java | 12 ++-- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/complex/BaseRepeatedValueVector.java b/java/vector/src/main/java/org/apache/arrow/vector/complex/BaseRepeatedValueVector.java index 7906d90c2f..7c4015299a 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/complex/BaseRepeatedValueVector.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/complex/BaseRepeatedValueVector.java @@ -355,16 +355,8 @@ public abstract class BaseRepeatedValueVector extends BaseValueVector implements offsetBuffer.getInt(index * OFFSET_WIDTH); } - /** Return if value at index is null (this implementation is always false). */ - @Override - public boolean isNull(int index) { -return false; - } - - /** Return if value at index is empty (this implementation is always false). */ - public boolean isEmpty(int index) { -return false; - } + /** Return if value at index is empty. */ + public abstract boolean isEmpty(int index); /** Starts a new repeated value. */ public int startNewValue(int index) {
(arrow-adbc) branch main updated: chore(dev/release): pin 15.0.2 for now (#1846)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 90a5654ff chore(dev/release): pin 15.0.2 for now (#1846) 90a5654ff is described below commit 90a5654ffbbe56626a54f2e13fd4508cdecc1f36 Author: David Li AuthorDate: Thu May 9 15:10:29 2024 +0900 chore(dev/release): pin 15.0.2 for now (#1846) Since (1) there is no arrow-c-glib for 16.0.0 yet and (2) for 16.0.0 we need to switch to pyarrow-all --- ci/conda_env_glib.txt | 2 +- ci/conda_env_python.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ci/conda_env_glib.txt b/ci/conda_env_glib.txt index 883ff7f92..5e3cd3266 100644 --- a/ci/conda_env_glib.txt +++ b/ci/conda_env_glib.txt @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. -arrow-c-glib +arrow-c-glib=15.0.2 glib gobject-introspection meson diff --git a/ci/conda_env_python.txt b/ci/conda_env_python.txt index 855496587..cf2abeeec 100644 --- a/ci/conda_env_python.txt +++ b/ci/conda_env_python.txt @@ -20,7 +20,7 @@ importlib-resources # nodejs is required by pyright nodejs >=13.0.0 pandas -pyarrow>=8.0.0 +pyarrow=15.0.2 pyright pytest setuptools
svn commit: r69046 - in /dev/arrow/apache-arrow-adbc-12-rc3: ./ apache-arrow-adbc-12.tar.gz apache-arrow-adbc-12.tar.gz.asc apache-arrow-adbc-12.tar.gz.sha512
Author: lidavidm Date: Thu May 9 03:41:42 2024 New Revision: 69046 Log: Apache Arrow ADBC 12 RC3 Added: dev/arrow/apache-arrow-adbc-12-rc3/ dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz (with props) dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.asc (with props) dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.sha512 Added: dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz -- svn:mime-type = application/gzip Added: dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.asc == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.asc -- svn:mime-type = application/pgp-signature Added: dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.sha512 == --- dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.sha512 (added) +++ dev/arrow/apache-arrow-adbc-12-rc3/apache-arrow-adbc-12.tar.gz.sha512 Thu May 9 03:41:42 2024 @@ -0,0 +1 @@ +a07d8b3557c196f23fd0e952c6d9d33d4ca3856ce0ff96e3316d1537bf4542d3c98462ec58bbad85b222fcd86242f050b86042a160bca772b7594ba37ba73464 apache-arrow-adbc-12.tar.gz
(arrow-adbc) annotated tag apache-arrow-adbc-12-rc3 updated (91804736a -> c2bff2924)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12-rc3 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12-rc3 was modified! *** from 91804736a (commit) to c2bff2924 (tag) tagging 91804736a5c478fcd9a69930c4bbef8f7af90a7f (commit) by David Li on Thu May 9 11:02:32 2024 +0900 - Log - ADBC Libraries 12 RC 3 --- No new revisions were added by this update. Summary of changes:
(arrow-adbc) branch maint-12 updated (a45f2dba7 -> 91804736a)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch maint-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git omit a45f2dba7 chore: update versions for 12 omit f447c741c chore: update CHANGELOG.md for 12 add f6541b2b9 docs: add warning (#1845) add 3e8ebcf10 chore: update CHANGELOG.md for 12 add 91804736a chore: update versions for 12 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a45f2dba7) \ N -- N -- N refs/heads/maint-12 (91804736a) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: ci/linux-packages/debian/changelog | 2 +- docs/source/driver/status.rst | 6 ++ 2 files changed, 7 insertions(+), 1 deletion(-)
(arrow-adbc) branch main updated: docs: add warning (#1845)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new f6541b2b9 docs: add warning (#1845) f6541b2b9 is described below commit f6541b2b932679a2045edb022ba565fa41cf7c79 Author: David Li AuthorDate: Thu May 9 11:01:54 2024 +0900 docs: add warning (#1845) Also see: #1841 --- docs/source/driver/status.rst | 6 ++ 1 file changed, 6 insertions(+) diff --git a/docs/source/driver/status.rst b/docs/source/driver/status.rst index b25f2f492..66fabff71 100644 --- a/docs/source/driver/status.rst +++ b/docs/source/driver/status.rst @@ -19,6 +19,12 @@ Driver Implementation Status +.. warning:: There is a known problem on macOS x86_64 when using two drivers + written in Go in the same process (unless working in a pure-Go + application), where using the second driver may crash. For more + details, see `GH-1841 + <https://github.com/apache/arrow-adbc/issues/1841>`_. + Implementation Status =
(arrow-adbc) annotated tag apache-arrow-adbc-12-rc2 updated (a45f2dba7 -> c6c0a06e0)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12-rc2 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12-rc2 was modified! *** from a45f2dba7 (commit) to c6c0a06e0 (tag) tagging a45f2dba74e96eafbe8834d948f9d7b4a1a1334a (commit) by David Li on Thu May 9 10:22:31 2024 +0900 - Log - ADBC Libraries 12 RC 2 --- No new revisions were added by this update. Summary of changes:
(arrow-adbc) branch maint-12 updated (d4e3f4f8e -> a45f2dba7)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch maint-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git omit d4e3f4f8e chore: update versions for 12 omit 623751a27 chore: update CHANGELOG.md for 12 add 62383368f chore(ci/linux-packages): fix RPM packages name (#1838) add 0fe5383ad fix(csharp/src/Apache.Arrow.Adbc/C): finalizer threw exception (#1842) add b615a7091 chore(ci/linux-packages): remove explicit arrow-glib-devel dependency (#1840) add f447c741c chore: update CHANGELOG.md for 12 add a45f2dba7 chore: update versions for 12 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d4e3f4f8e) \ N -- N -- N refs/heads/maint-12 (a45f2dba7) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: CHANGELOG.md | 3 +- ci/linux-packages/Rakefile | 22 +++- ci/linux-packages/debian/changelog | 2 +- ci/linux-packages/yum/apache-arrow-adbc.spec.in| 65 +++--- .../src/Apache.Arrow.Adbc/C/CAdbcDriverImporter.cs | 15 - 5 files changed, 66 insertions(+), 41 deletions(-)
(arrow) branch main updated: GH-41573: [Java] VectorSchemaRoot uses inefficient stream to copy fieldVectors (#41574)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow.git The following commit(s) were added to refs/heads/main by this push: new 7bfe02db04 GH-41573: [Java] VectorSchemaRoot uses inefficient stream to copy fieldVectors (#41574) 7bfe02db04 is described below commit 7bfe02db04e34fc1ab6df6f647a76899e0c654db Author: David Schlosnagle AuthorDate: Wed May 8 19:46:15 2024 -0400 GH-41573: [Java] VectorSchemaRoot uses inefficient stream to copy fieldVectors (#41574) ### Rationale for this change While reviewing allocation profiling of an Arrow intensive application, I noticed significant allocations due to `ArrayList#grow()` originating from `org.apache.arrow.vector.VectorSchemaRoot#getFieldVectors()`. The `org.apache.arrow.vector.VectorSchemaRoot#getFieldVectors()` method uses an inefficient `fieldVectors.stream().collect(Collectors.toList())` to create a list copy, leading to reallocations as the target list is collected. This could be replaced with a more efficent `new Arr [...] ### What changes are included in this PR? * Use `Collections.unmodifiableList(List)` to return unmodifiable list view of `fieldVectors` from `getFieldVectors()` * Pre-size the `fieldVectors` `ArrayList` in static factory `VectorSchemaRoot#create(Schema, BufferAllocator)` * `VectorSchemaRoot#setRowCount(int)` iterates over instance `fieldVectors` instead of copied list (similar to existing `allocateNew()`, `clear()`, `contentToTSVString()`). ### Are these changes tested? These changes are covered by existing unit and integration tests. ### Are there any user-facing changes? No * GitHub Issue: #41573 Authored-by: David Schlosnagle Signed-off-by: David Li --- .../src/main/java/org/apache/arrow/vector/VectorSchemaRoot.java| 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/java/vector/src/main/java/org/apache/arrow/vector/VectorSchemaRoot.java b/java/vector/src/main/java/org/apache/arrow/vector/VectorSchemaRoot.java index 8768a90c80..9a92ce5060 100644 --- a/java/vector/src/main/java/org/apache/arrow/vector/VectorSchemaRoot.java +++ b/java/vector/src/main/java/org/apache/arrow/vector/VectorSchemaRoot.java @@ -19,6 +19,7 @@ package org.apache.arrow.vector; import java.util.ArrayList; import java.util.Arrays; +import java.util.Collections; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; @@ -121,7 +122,7 @@ public class VectorSchemaRoot implements AutoCloseable { * Creates a new set of empty vectors corresponding to the given schema. */ public static VectorSchemaRoot create(Schema schema, BufferAllocator allocator) { -List fieldVectors = new ArrayList<>(); +List fieldVectors = new ArrayList<>(schema.getFields().size()); for (Field field : schema.getFields()) { FieldVector vector = field.createVector(allocator); fieldVectors.add(vector); @@ -160,7 +161,7 @@ public class VectorSchemaRoot implements AutoCloseable { } public List getFieldVectors() { -return fieldVectors.stream().collect(Collectors.toList()); +return Collections.unmodifiableList(fieldVectors); } /** @@ -236,7 +237,7 @@ public class VectorSchemaRoot implements AutoCloseable { */ public void setRowCount(int rowCount) { this.rowCount = rowCount; -for (FieldVector v : getFieldVectors()) { +for (FieldVector v : fieldVectors) { v.setValueCount(rowCount); } }
(arrow-adbc) annotated tag apache-arrow-adbc-12-rc1 updated (d4e3f4f8e -> 2493a3e3b)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12-rc1 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12-rc1 was modified! *** from d4e3f4f8e (commit) to 2493a3e3b (tag) tagging d4e3f4f8e3cceb32c6de98a53ef4fc5dcffc8751 (commit) by David Li on Wed May 8 16:23:54 2024 +0900 - Log - ADBC Libraries 12 RC 1 --- No new revisions were added by this update. Summary of changes:
(arrow-adbc) branch maint-12 updated (5bd321355 -> d4e3f4f8e)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch maint-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git omit 5bd321355 chore: update versions for 12 omit de4c82842 chore: update CHANGELOG.md for 12 add 2d842cbae fix(dev/release): handle versioning scheme in binary verification (#1834) add 10fb52767 ci: re-enable arm64v8 build (#1836) add 623751a27 chore: update CHANGELOG.md for 12 add d4e3f4f8e chore: update versions for 12 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (5bd321355) \ N -- N -- N refs/heads/maint-12 (d4e3f4f8e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .github/workflows/packaging.yml | 3 +- CHANGELOG.md| 43 + ci/linux-packages/debian/changelog | 2 +- ci/linux-packages/yum/apache-arrow-adbc.spec.in | 2 +- dev/release/verify-release-candidate.sh | 22 + 5 files changed, 41 insertions(+), 31 deletions(-)
(arrow-adbc) branch main updated: ci: re-enable arm64v8 build (#1836)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 10fb52767 ci: re-enable arm64v8 build (#1836) 10fb52767 is described below commit 10fb52767b1324fa412d06eabef353585f56066f Author: David Li AuthorDate: Wed May 8 16:22:47 2024 +0900 ci: re-enable arm64v8 build (#1836) Fixes #1835. --- .github/workflows/packaging.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.github/workflows/packaging.yml b/.github/workflows/packaging.yml index 4fa2a7911..6b26e9920 100644 --- a/.github/workflows/packaging.yml +++ b/.github/workflows/packaging.yml @@ -558,8 +558,7 @@ jobs: strategy: fail-fast: false matrix: -# XXX(lidavidm): disable arm64 build for now while I test things -arch: ["amd64"] +arch: ["amd64", "arm64v8"] manylinux_version: ["2014"] is_pr: - ${{ startsWith(github.ref, 'refs/pull/') }}
svn commit: r69024 - in /dev/arrow/apache-arrow-adbc-12-rc0: ./ apache-arrow-adbc-12.tar.gz apache-arrow-adbc-12.tar.gz.asc apache-arrow-adbc-12.tar.gz.sha512
Author: lidavidm Date: Tue May 7 23:42:53 2024 New Revision: 69024 Log: Apache Arrow ADBC 12 RC0 Added: dev/arrow/apache-arrow-adbc-12-rc0/ dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz (with props) dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.asc (with props) dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.sha512 Added: dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz -- svn:mime-type = application/gzip Added: dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.asc == Binary file - no diff available. Propchange: dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.asc -- svn:mime-type = application/pgp-signature Added: dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.sha512 == --- dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.sha512 (added) +++ dev/arrow/apache-arrow-adbc-12-rc0/apache-arrow-adbc-12.tar.gz.sha512 Tue May 7 23:42:53 2024 @@ -0,0 +1 @@ +63bdf377802f9b58920e8f56fe54accfc132c2c2823587726c499a896c4c360decc308a40034a660a2fbc073ed649c056e2a9b7317fb72e23f9122d70edec727 apache-arrow-adbc-12.tar.gz
(arrow-adbc) branch maint-12 created (now 5bd321355)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch maint-12 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git at 5bd321355 chore: update versions for 12 No new revisions were added by this update.
(arrow-adbc) annotated tag apache-arrow-adbc-12-rc0 updated (5bd321355 -> bdda59066)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to annotated tag apache-arrow-adbc-12-rc0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git *** WARNING: tag apache-arrow-adbc-12-rc0 was modified! *** from 5bd321355 (commit) to bdda59066 (tag) tagging 5bd32135505b63fb8fb0ec617ae7c1f2c9e66bfb (commit) by David Li on Wed May 8 08:12:19 2024 +0900 - Log - ADBC Libraries 12 RC 0 --- No new revisions were added by this update. Summary of changes:
(arrow-adbc) branch gh-1144 deleted (was 3bbe58648)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch gh-1144 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 3bbe58648 Fix The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated: feat(go/adbc/driver/snowflake): support parameter binding (#1808)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 8f0bdb367 feat(go/adbc/driver/snowflake): support parameter binding (#1808) 8f0bdb367 is described below commit 8f0bdb367708f36183e7aa3691ff14c17c28d4e2 Author: David Li AuthorDate: Tue May 7 23:11:25 2024 +0900 feat(go/adbc/driver/snowflake): support parameter binding (#1808) Fixes #1144. --- c/driver/snowflake/snowflake_test.cc | 2 +- go/adbc/driver/snowflake/binding.go | 153 ++ go/adbc/driver/snowflake/concat_reader.go | 107 + go/adbc/driver/snowflake/driver.go| 2 +- go/adbc/driver/snowflake/statement.go | 56 ++- go/adbc/go.mod| 4 +- go/adbc/go.sum| 8 +- 7 files changed, 321 insertions(+), 11 deletions(-) diff --git a/c/driver/snowflake/snowflake_test.cc b/c/driver/snowflake/snowflake_test.cc index 0fe07ecbd..a4d742491 100644 --- a/c/driver/snowflake/snowflake_test.cc +++ b/c/driver/snowflake/snowflake_test.cc @@ -146,7 +146,7 @@ class SnowflakeQuirks : public adbc_validation::DriverQuirks { bool supports_metadata_current_catalog() const override { return false; } bool supports_metadata_current_db_schema() const override { return false; } bool supports_partitioned_data() const override { return false; } - bool supports_dynamic_parameter_binding() const override { return false; } + bool supports_dynamic_parameter_binding() const override { return true; } bool supports_error_on_incompatible_schema() const override { return false; } bool ddl_implicit_commit_txn() const override { return true; } std::string db_schema() const override { return schema_; } diff --git a/go/adbc/driver/snowflake/binding.go b/go/adbc/driver/snowflake/binding.go new file mode 100644 index 0..7f6878945 --- /dev/null +++ b/go/adbc/driver/snowflake/binding.go @@ -0,0 +1,153 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package snowflake + +import ( + "database/sql" + "database/sql/driver" + "fmt" + "io" + + "github.com/apache/arrow-adbc/go/adbc" + "github.com/apache/arrow/go/v17/arrow" + "github.com/apache/arrow/go/v17/arrow/array" +) + +func convertArrowToNamedValue(batch arrow.Record, index int) ([]driver.NamedValue, error) { + // see goTypeToSnowflake in gosnowflake + // technically, snowflake can bind an array of values at once, but + // only for INSERT, so we can't take advantage of that without + // analyzing the query ourselves + params := make([]driver.NamedValue, batch.NumCols()) + for i, field := range batch.Schema().Fields() { + rawColumn := batch.Column(i) + params[i].Ordinal = i + 1 + switch column := rawColumn.(type) { + case *array.Boolean: + params[i].Value = sql.NullBool{ + Bool: column.Value(index), + Valid: column.IsValid(index), + } + case *array.Float32: + // Snowflake only recognizes float64 + params[i].Value = sql.NullFloat64{ + Float64: float64(column.Value(index)), + Valid: column.IsValid(index), + } + case *array.Float64: + params[i].Value = sql.NullFloat64{ + Float64: column.Value(index), + Valid: column.IsValid(index), + } + case *array.Int8: + // Snowflake only recognizes int64 + params[i].Value = sql.NullInt64{ + Int64: int64(column.Value(index)), + Valid: column.IsValid(index), +
(arrow-adbc) branch dependabot/maven/java/com.uber.nullaway-nullaway-0.10.26 deleted (was 8c9df91a2)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/com.uber.nullaway-nullaway-0.10.26 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 8c9df91a2 chore(java): bump com.uber.nullaway:nullaway in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated (7b16e476f -> 00aa664d4)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from 7b16e476f chore(java): bump org.checkerframework:checker-qual from 3.42.0 to 3.43.0 in /java (#1824) add 00aa664d4 chore(java): bump com.uber.nullaway:nullaway from 0.10.25 to 0.10.26 in /java (#1825) No new revisions were added by this update. Summary of changes: java/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow-adbc) branch dependabot/maven/java/org.checkerframework-checker-qual-3.43.0 deleted (was 8791a54c1)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.checkerframework-checker-qual-3.43.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 8791a54c1 chore(java): bump org.checkerframework:checker-qual in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated: chore(java): bump org.checkerframework:checker-qual from 3.42.0 to 3.43.0 in /java (#1824)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 7b16e476f chore(java): bump org.checkerframework:checker-qual from 3.42.0 to 3.43.0 in /java (#1824) 7b16e476f is described below commit 7b16e476f64313ededb6f8ea28b484ccc59a7254 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon May 6 23:11:45 2024 -0400 chore(java): bump org.checkerframework:checker-qual from 3.42.0 to 3.43.0 in /java (#1824) Bumps [org.checkerframework:checker-qual](https://github.com/typetools/checker-framework) from 3.42.0 to 3.43.0. Release notes Sourced from https://github.com/typetools/checker-framework/releases;>org.checkerframework:checker-qual's releases. Checker Framework 3.43.0 Version 3.43.0 (May 1, 2024) User-visible changes: Method, constructor, lambda, and method reference type inference has been greatly improved. The -AconservativeUninferredTypeArguments option is no longer necessary and has been removed. Renamed command-line arguments: -AskipDirs has been renamed to -AskipFiles. -AskipDirs will continue to work for the time being. New command-line arguments: -AonlyFiles complements -AskipFiles A specialized inference algorithm for the Resource Leak Checker runs automatically as part of whole-program inference. Implementation details: Deprecated ObjectCreationNode#getConstructor in favor of new ObjectCreationNode#getTypeToInstantiate(). Renamed AbstractCFGVisualizer.visualizeBlockHelper() to visualizeBlockWithSeparator(). Moved methods from TreeUtils to subclasses of TreeUtilsAfterJava11: isConstantCaseLabelTree isDefaultCaseLabelTree isPatternCaseLabelTree Renamed BaseTypeVisitor.checkForPolymorphicQualifiers() to warnInvalidPolymorphicQualifier(). Closed issues: https://redirect.github.com/typetools/checker-framework/issues/979;>#979, https://redirect.github.com/typetools/checker-framework/issues/4559;>#4559, https://redirect.github.com/typetools/checker-framework/issues/4593;>#4593, https://redirect.github.com/typetools/checker-framework/issues/5058;>#5058, https://redirect.github.com/typetools/checker-framework/issues/5734;>#5734, https://redirect.github.com/typetools/checker-framework/issues/5781;>#5781, https://redirect.github.com/typetools/checker-framework/issues/6071;>#6071, https://redirect.github.com/typetools/checker-framework/issues/6093;>#6093, https://redirect.github.com/typetools/checker-framework/issues/6239;>#6239, https://redirect.github.com/typetools/checker-framework/issues/6297;>#6297, https://redirect.github.com/typetools/checker-framework/issues/6317;>#6317, https://redirect.github.com/typetools/checker-framework/issues/6322;>#6322, https://redirect.github.com/typetools/checker-framework/issues/6346;>#6346, https://redirect.github.com/typetools/checker-framework/issues/6373;>#6373, https://redirect.github.com/typetools/checker-framework/issues/6376;>#6376, https://redirect.github.com/typetools/checker-framework/issues/6378;>#6378, https://redirect.github.com/typetools/checker-framework/issues/6379;>#6379, https://redirect.github.com/typetools/checker-framework/issues/6380;>#6380, https://redirect.github.com/typetools/checker-framework/issues/6389;>#6389, https://redirect.github.com/typetools/checker-framework/issues/6393;>#6393, https://redirect.github.com/typetools/checker-framework/issues/6396;>#6396, https://redirect.github.com/typetools/checker-framework/issues/6402;>#6402, https://redirect.github.com/typetools/checker-framework/issues/6406;>#6406, https://redirect.github.com/typetools/checker-framework/issues/6407;>#6407, https://redirect.github.com/typetools/checker-framework/issues/6417;>#6417, https://redirect.github.com/typetools/checker-framework/issues/6421;>#6421, https://redirect.github.com/typetools/checker-framework/issues/6430;>#6430, https://redirect.github.com/typetools/checker-framework/issues/6433;>#6433, https://redirect.github.com/typetools/checker-framework/issues/6438;>#6438, https://redirect.github.com/typetools/checker-framework/issues/6442;>#6442, https://redirect.github.com/typetools/checker-framework/issues/6473;>#6473, https://redirect.github.com/typetools/checker-framework/issues/6480;>#6480, https://redirect.github.com/typetools/checker-framework/issues/6507;>#6507, https://redirect.github.com/typetools/checker-framework/issues/6531;>#6531, https://redirect.github.com/typetools/checker-f
(arrow-adbc) branch dependabot/maven/java/org.checkerframework-checker-3.43.0 deleted (was 3b81bc4bb)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/org.checkerframework-checker-3.43.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 3b81bc4bb chore(java): bump org.checkerframework:checker in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated (af422c1d1 -> 6ce5d7f02)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from af422c1d1 chore(java): bump com.google.errorprone:error_prone_core from 2.27.0 to 2.27.1 in /java (#1826) add 6ce5d7f02 chore(java): bump org.checkerframework:checker from 3.42.0 to 3.43.0 in /java (#1823) No new revisions were added by this update. Summary of changes: java/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow-adbc) branch dependabot/maven/java/com.google.errorprone-error_prone_core-2.27.1 deleted (was db8161463)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/maven/java/com.google.errorprone-error_prone_core-2.27.1 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was db8161463 chore(java): bump com.google.errorprone:error_prone_core in /java The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated (129785a44 -> af422c1d1)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from 129785a44 chore(go/adbc): bump golang.org/x/tools from 0.20.0 to 0.21.0 in /go/adbc (#1828) add af422c1d1 chore(java): bump com.google.errorprone:error_prone_core from 2.27.0 to 2.27.1 in /java (#1826) No new revisions were added by this update. Summary of changes: java/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(arrow-adbc) branch dependabot/go_modules/go/adbc/golang.org/x/tools-0.21.0 deleted (was 070ed912a)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch dependabot/go_modules/go/adbc/golang.org/x/tools-0.21.0 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 070ed912a chore(go/adbc): bump golang.org/x/tools in /go/adbc The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(arrow-adbc) branch main updated: chore(go/adbc): bump golang.org/x/tools from 0.20.0 to 0.21.0 in /go/adbc (#1828)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 129785a44 chore(go/adbc): bump golang.org/x/tools from 0.20.0 to 0.21.0 in /go/adbc (#1828) 129785a44 is described below commit 129785a44af4f6e879101234651f9b4560dc3627 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon May 6 19:11:30 2024 -0400 chore(go/adbc): bump golang.org/x/tools from 0.20.0 to 0.21.0 in /go/adbc (#1828) Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.20.0 to 0.21.0. Commits https://github.com/golang/tools/commit/cc29c91ba3acb47c1793da56d919cc40def4ea41;>cc29c91 go.mod: update golang.org/x dependencies https://github.com/golang/tools/commit/397fef9629377c5179518414e4502009dc0918a2;>397fef9 gopls/internal/protocol: add links to LSP spec https://github.com/golang/tools/commit/e2a352c850a52c32b9bfdb78fbfb97882b7c1424;>e2a352c internal/refactor/inline: extensible API https://github.com/golang/tools/commit/c16c816ac486df69471674bcaf6f1a8f76f2bdcf;>c16c816 go/analysis/passes/stdversion: test *.go go.mod version https://github.com/golang/tools/commit/629a7be6d0a4808cd8e7af805986652d81bb975f;>629a7be go/analysis/analysistest: stricter errors and GOWORK setting https://github.com/golang/tools/commit/4db16973aa22c2c1a5a873261a68949f351efe2b;>4db1697 go/packages/packagestest: fold modules_111.go into modules.go https://github.com/golang/tools/commit/ccdef3cc2699ea2a30a2931a43a1774bd974000a;>ccdef3c gopls/internal/golang: fix nil panic in InlayHint https://github.com/golang/tools/commit/74c9cfe4d22faa696baabeea02df6493b15e8c79;>74c9cfe go/analysis: add Pass.ReadFile https://github.com/golang/tools/commit/5ef4fc9014e76bcc56324b24218c0216a7146306;>5ef4fc9 gopls/internal/golang/completion: fix the isEmptyInterface predicate https://github.com/golang/tools/commit/77f691badad098462fed0a46c9a3f5f3d178b633;>77f691b internal/gcimporter: use Alias.Rhs, not unsafe hack Additional commits viewable in https://github.com/golang/tools/compare/v0.20.0...v0.21.0;>compare view [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/tools=go_modules=0.20.0=0.21.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- go/adbc/go.mod | 12 ++-- go/adbc/go.sum | 24 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/go/adbc/go.mod b/go/adbc/go.mod index 0c22e2b93..05dd4871a 100644 --- a/go/adbc/go.mod +++ b/go/adbc/go.mod @@ -29,7 +29,7 @@ require ( github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a golang.org/x/exp v0.0.0-20240318143956-a85f2c67cd81 golang.org/x/s
(arrow-adbc) branch gh-1144 updated (b78fed176 -> 3bbe58648)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch gh-1144 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from b78fed176 Fix add 3bbe58648 Fix No new revisions were added by this update. Summary of changes: go/adbc/driver/snowflake/binding.go | 20 go/adbc/driver/snowflake/concat_reader.go | 5 - go/adbc/driver/snowflake/statement.go | 7 --- 3 files changed, 24 insertions(+), 8 deletions(-)
(arrow-adbc) branch main updated: fix(csharp): Change option translation to be case-sensitive (#1820)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new b2c34e1c6 fix(csharp): Change option translation to be case-sensitive (#1820) b2c34e1c6 is described below commit b2c34e1c6a9d835de72e5b5595a7a5b5eebd1bd9 Author: Curt Hagenlocher AuthorDate: Sun May 5 01:29:40 2024 -0700 fix(csharp): Change option translation to be case-sensitive (#1820) Closes #1819 --- csharp/src/Apache.Arrow.Adbc/AdbcOptions.cs | 37 ++--- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/csharp/src/Apache.Arrow.Adbc/AdbcOptions.cs b/csharp/src/Apache.Arrow.Adbc/AdbcOptions.cs index b5a54b240..92390f7b6 100644 --- a/csharp/src/Apache.Arrow.Adbc/AdbcOptions.cs +++ b/csharp/src/Apache.Arrow.Adbc/AdbcOptions.cs @@ -54,9 +54,12 @@ namespace Apache.Arrow.Adbc public static string GetEnabled(bool value) => value ? Enabled : Disabled; public static bool GetEnabled(string value) { -if (StringComparer.OrdinalIgnoreCase.Equals(value, Enabled)) { return true; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, Disabled)) { return false; } -throw new NotSupportedException("unknown enabled flag"); +return value switch +{ +Enabled => true, +Disabled => false, +_ => throw new NotSupportedException("unknown enabled flag"), +}; } public static string GetIsolationLevel(IsolationLevel value) @@ -76,14 +79,17 @@ namespace Apache.Arrow.Adbc public static IsolationLevel GetIsolationLevel(string value) { -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.Default)) { return Adbc.IsolationLevel.Default; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.ReadUncommitted)) { return Adbc.IsolationLevel.ReadUncommitted; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.ReadCommitted)) { return Adbc.IsolationLevel.ReadCommitted; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.RepeatableRead)) { return Adbc.IsolationLevel.RepeatableRead; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.Snapshot)) { return Adbc.IsolationLevel.Snapshot; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.Serializable)) { return Adbc.IsolationLevel.Serializable; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IsolationLevels.Linearizable)) { return Adbc.IsolationLevel.Linearizable; } -throw new NotSupportedException("unknown isolation level"); +return value switch +{ +IsolationLevels.Default => Adbc.IsolationLevel.Default, +IsolationLevels.ReadUncommitted => Adbc.IsolationLevel.ReadUncommitted, +IsolationLevels.ReadCommitted => Adbc.IsolationLevel.ReadCommitted, +IsolationLevels.RepeatableRead => Adbc.IsolationLevel.RepeatableRead, +IsolationLevels.Snapshot => Adbc.IsolationLevel.Snapshot, +IsolationLevels.Serializable => Adbc.IsolationLevel.Serializable, +IsolationLevels.Linearizable => Adbc.IsolationLevel.Linearizable, +_ => throw new NotSupportedException("unknown isolation level"), +}; } public static string GetIngestMode(BulkIngestMode value) @@ -98,9 +104,12 @@ namespace Apache.Arrow.Adbc public static BulkIngestMode GetIngestMode(string value) { -if (StringComparer.OrdinalIgnoreCase.Equals(value, IngestMode.Create)) { return BulkIngestMode.Create; } -if (StringComparer.OrdinalIgnoreCase.Equals(value, IngestMode.Append)) { return BulkIngestMode.Append; } -throw new NotSupportedException("unknown ingestion mode"); +return value switch +{ +IngestMode.Create => BulkIngestMode.Create, +IngestMode.Append => BulkIngestMode.Append, +_ => throw new NotSupportedException("unknown ingestion mode"), +}; } } }
(arrow-adbc) branch gh-1144 updated (c1342955d -> b78fed176)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch gh-1144 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git from c1342955d feat(go/adbc/driver/snowflake): support parameter binding add b78fed176 Fix No new revisions were added by this update. Summary of changes: go/adbc/driver/snowflake/concat_reader.go | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
(arrow-adbc) branch gh-1144 updated (aeca9c9af -> c1342955d)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch gh-1144 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git discard aeca9c9af Merge branch 'gh-1804' into gh-1144 discard 1d0fa3b6b Remove redundant release discard ef3459d25 Bump arrow-go version discard 3e208f548 More fixes omit 928726beb Simplify discard 22c5d4a3a feat(go/adbc/driver/snowflake): support parameter binding omit 775fde2f1 fix(go/adbc/driver/snowflake): handle empty result sets add a48c3b050 feat(csharp): imported objects should have call "release" when no longer in use (#1802) add a830bf26c fix(csharp): an assortment of small fixes not worth individual pull requests (#1807) add 2d6b06b4f ci: don't assign milestone for draft PRs (#1810) add d426d8027 chore(r): Synchronize vendoring scripts with latest changes (#1809) add 2e044e85a fix(go/adbc/driver/snowflake): handle empty result sets (#1805) add 8ac730436 fix(csharp/src/Apache.Arrow.Adbc/C): correctly handle null driver entries for imported drivers (#1812) add c1342955d feat(go/adbc/driver/snowflake): support parameter binding This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (aeca9c9af) \ N -- N -- N refs/heads/gh-1144 (c1342955d) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .github/workflows/dev_pr.yml | 2 +- csharp/src/Apache.Arrow.Adbc/AdbcException.cs | 5 + .../src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs | 109 +--- .../src/Apache.Arrow.Adbc/C/CAdbcDriverImporter.cs | 668 ++--- csharp/src/Apache.Arrow.Adbc/C/Delegates.cs| 51 ++ csharp/src/Apache.Arrow.Adbc/C/NativeDelegate.cs | 14 +- .../Extensions/CollectionExtensions.cs | 2 +- csharp/src/Client/AdbcCommand.cs | 49 +- csharp/src/Client/AdbcConnection.cs| 22 +- csharp/src/Client/AdbcDataReader.cs| 2 +- csharp/test/Drivers/Interop/Snowflake/CastTests.cs | 2 +- .../test/Drivers/Interop/Snowflake/ClientTests.cs | 2 +- .../test/Drivers/Interop/Snowflake/DriverTests.cs | 2 +- go/adbc/driver/snowflake/driver_test.go| 19 + .../src/c/vendor/fmt/include/fmt/.gitignore| 1 + r/adbcsqlite/configure | 6 +- r/adbcsqlite/src/Makevars.in | 2 +- r/adbcsqlite/src/c/driver/framework/.gitignore | 4 +- .../src/c/vendor/fmt/include/fmt/.gitignore| 1 + r/adbcsqlite/tools/.gitignore | 19 - 20 files changed, 746 insertions(+), 236 deletions(-) create mode 100644 csharp/src/Apache.Arrow.Adbc/C/Delegates.cs delete mode 100644 r/adbcsqlite/tools/.gitignore
(arrow-adbc) branch main updated: fix(csharp/src/Apache.Arrow.Adbc/C): correctly handle null driver entries for imported drivers (#1812)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git The following commit(s) were added to refs/heads/main by this push: new 8ac730436 fix(csharp/src/Apache.Arrow.Adbc/C): correctly handle null driver entries for imported drivers (#1812) 8ac730436 is described below commit 8ac730436474b48cccebb58f79048cba8b0d8adf Author: Curt Hagenlocher AuthorDate: Sat May 4 04:22:24 2024 -0700 fix(csharp/src/Apache.Arrow.Adbc/C): correctly handle null driver entries for imported drivers (#1812) Validates that required function pointers are present in an imported driver and backfills optional functions to return ADBC_STATUS_NOT_IMPLEMENTED. Closes #1811 --- csharp/src/Apache.Arrow.Adbc/AdbcException.cs | 5 + .../src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs | 10 +- .../src/Apache.Arrow.Adbc/C/CAdbcDriverImporter.cs | 248 - csharp/src/Apache.Arrow.Adbc/C/Delegates.cs| 8 +- 4 files changed, 263 insertions(+), 8 deletions(-) diff --git a/csharp/src/Apache.Arrow.Adbc/AdbcException.cs b/csharp/src/Apache.Arrow.Adbc/AdbcException.cs index 98faefa6c..c4deada39 100644 --- a/csharp/src/Apache.Arrow.Adbc/AdbcException.cs +++ b/csharp/src/Apache.Arrow.Adbc/AdbcException.cs @@ -56,6 +56,11 @@ namespace Apache.Arrow.Adbc return new AdbcException(message, AdbcStatusCode.NotImplemented); } +internal static AdbcException Missing(string name) +{ +return new AdbcException($"Driver does not implement required function Adbc{name}", AdbcStatusCode.InternalError); +} + /// /// For database providers which support it, contains a standard /// SQL 5-character return code indicating the success or failure diff --git a/csharp/src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs b/csharp/src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs index 7352e3480..2b77315f2 100644 --- a/csharp/src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs +++ b/csharp/src/Apache.Arrow.Adbc/C/CAdbcDriverExporter.cs @@ -30,7 +30,7 @@ namespace Apache.Arrow.Adbc.C private static unsafe readonly NativeDelegate s_releaseError = new NativeDelegate(ReleaseError); #if NET5_0_OR_GREATER -private static unsafe delegate* unmanaged ReleaseErrorPtr => (delegate* unmanaged)s_releaseError.Pointer; +internal static unsafe delegate* unmanaged ReleaseErrorPtr => (delegate* unmanaged)s_releaseError.Pointer; private static unsafe delegate* unmanaged ReleaseDriverPtr => private static unsafe delegate* unmanaged ReleasePartitionsPtr => @@ -60,7 +60,7 @@ namespace Apache.Arrow.Adbc.C private static unsafe delegate* unmanaged StatementSetSubstraitPlanPtr => private static unsafe delegate* unmanaged StatementGetParameterSchemaPtr => #else -private static unsafe IntPtr ReleaseErrorPtr => s_releaseError.Pointer; +internal static unsafe IntPtr ReleaseErrorPtr => s_releaseError.Pointer; private static unsafe IntPtr ReleaseDriverPtr = NativeDelegate.AsNativePointer(ReleaseDriver); private static unsafe IntPtr ReleasePartitionsPtr = NativeDelegate.AsNativePointer(ReleasePartitions); @@ -84,8 +84,8 @@ namespace Apache.Arrow.Adbc.C private static unsafe IntPtr StatementExecuteQueryPtr = NativeDelegate.AsNativePointer(ExecuteStatementQuery); private static unsafe IntPtr StatementExecutePartitionsPtr = NativeDelegate.AsNativePointer(ExecuteStatementPartitions); private static unsafe IntPtr StatementNewPtr = NativeDelegate.AsNativePointer(NewStatement); -private static unsafe IntPtr StatementReleasePtr = NativeDelegate.AsNativePointer(ReleaseStatement); -private static unsafe IntPtr StatementPreparePtr = NativeDelegate.AsNativePointer(PrepareStatement); +private static unsafe IntPtr StatementReleasePtr = NativeDelegate.AsNativePointer(ReleaseStatement); +private static unsafe IntPtr StatementPreparePtr = NativeDelegate.AsNativePointer(PrepareStatement); private static unsafe IntPtr StatementSetSqlQueryPtr = NativeDelegate.AsNativePointer(SetStatementSqlQuery); private static unsafe IntPtr StatementSetSubstraitPlanPtr = NativeDelegate.AsNativePointer(SetStatementSubstraitPlan); private static unsafe IntPtr StatementGetParameterSchemaPtr = NativeDelegate.AsNativePointer(GetStatementParameterSchema); @@ -129,7 +129,7 @@ namespace Apache.Arrow.Adbc.C return 0; } -private unsafe static void ReleaseError(CAdbcError* error) +internal unsafe static void ReleaseError(CAdbcError* error) { if (error != null && ((IntPtr)error->message) != IntPtr.Zero) { diff --git a/csharp/src/Apache.Arrow.Adbc/C/CAdbcDriver
(arrow-adbc) branch gh-1804 deleted (was 5532265b0)
This is an automated email from the ASF dual-hosted git repository. lidavidm pushed a change to branch gh-1804 in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git was 5532265b0 Add pure Go test The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.