user  

Messages by Thread

• Spark 3.3.0/3.2.2: java.io.IOException: can not read class org.apache.parquet.format.PageHeader: don't know what type: 15 FengYu Cao
  • Re: Spark 3.3.0/3.2.2: java.io.IOException: can not read class org.apache.parquet.format.PageHeader: don't know what type: 15 Chao Sun
  • Re: Spark 3.3.0/3.2.2: java.io.IOException: can not read class org.apache.parquet.format.PageHeader: don't know what type: 15 FengYu Cao
• Moving to Spark 3x from Spark2 rajat kumar
  • Re: Moving to Spark 3x from Spark2 Khalid Mammadov
  • Re: Moving to Spark 3x from Spark2 Martin Andersson
• deciding Spark tasks & optimization resource rajat kumar
  • Re: deciding Spark tasks & optimization resource Gibson
• Spark 3.3.0 with Structure Streaming from Kafka Issue on commons-pools2 Raymond Tang
• Spark SQL Predict Pushdown for Hive Bucketed Table Raymond Tang
• Structured Streaming - data not being read (offsets not getting committed ?) karan alang
• 回复：Re: Spark got incorrect scala version while using spark 3.2.1 and spark 3.2.2 ckgppl_yan
• Spark got incorrect scala version while using spark 3.2.1 and spark 3.2.2 ckgppl_yan
  • Re: Spark got incorrect scala version while using spark 3.2.1 and spark 3.2.2 Sean Owen
  • Re: Spark got incorrect scala version while using spark 3.2.1 and spark 3.2.2 pengyh
• Profiling PySpark Pandas UDF Subash Prabanantham
  • Re: Profiling PySpark Pandas UDF Gourav Sengupta
  • Re: Profiling PySpark Pandas UDF Andrew Melo
  • Re: Profiling PySpark Pandas UDF Sean Owen
  • Re: Profiling PySpark Pandas UDF Russell Jurney
  • Re: Profiling PySpark Pandas UDF Takuya UESHIN
  • Re: Profiling PySpark Pandas UDF Sean Owen
  • Re: Profiling PySpark Pandas UDF Russell Jurney
  • Re: Profiling PySpark Pandas UDF Subash Prabanantham
  • Re: Profiling PySpark Pandas UDF Abdeali Kothari
  • RE: Profiling PySpark Pandas UDF Luca Canali
  • Re: Profiling PySpark Pandas UDF Abdeali Kothari
  • RE: Profiling PySpark Pandas UDF Luca Canali
  • Re: Profiling PySpark Pandas UDF Gourav Sengupta
• spark-3.2.2-bin-without-hadoop : NoClassDefFoundError: org/apache/log4j/spi/Filter when starting the master FLORANCE Grégory
  • Re: spark-3.2.2-bin-without-hadoop : NoClassDefFoundError: org/apache/log4j/spi/Filter when starting the master Sean Owen
• Question regarding checkpointing with kafka structured streaming Martin Andersson
• [Spark SQL]: Does Spark preserve the order in a nested ORDER BY? Vinay Londhe
• Filtering by job group in the Spark UI / API Yeachan Park
• Spark streaming Prajith Vellukkai
  • Re: Spark streaming ミユナ (alice)
  • Spark streaming sandra sukumaran
  • Re: Spark streaming Ajit Kumar Amit
  • Re: [EXTERNAL] Re: Spark streaming Saurabh Gulati
  • Re: [EXTERNAL] Re: Spark streaming sandra sukumaran
  • Re: Spark streaming Gourav Sengupta
• Data ingestion Akash Vellukai
  • Re: Data ingestion Pasha Finkelshtein
  • Re: Data ingestion Yuri Oleynikov (‫יורי אולייניקוב‬‎)
  • Re: Data ingestion pengyh
  • Re: Data ingestion Pasha Finkelshtein
• Spark streaming - Data Ingestion Akash Vellukai
  • Re: Spark streaming - Data Ingestion Gibson
  • Re: [EXTERNAL] Re: Spark streaming - Data Ingestion Saurabh Gulati
  • Re: [EXTERNAL] Re: Spark streaming - Data Ingestion Akash Vellukai
  • Re: [EXTERNAL] Re: Spark streaming - Data Ingestion Gibson
• Supported Hadoop versions for Spark 3.3 Håkan Nordgren
  • Re: Supported Hadoop versions for Spark 3.3 pengyh
• PySpark schema sanitization Shay Elbaz
  • Unsubscribe Peter Kovgan
• Spark with GPU rajat kumar
  • Re: Spark with GPU Sean Owen
  • Re: Spark with GPU rajat kumar
  • Re: Spark with GPU Sean Owen
  • Re: Spark with GPU Alessandro Bellina
  • Re: Spark with GPU Gourav Sengupta
  • Spark with GPU Irene Markelic
  • Re: Spark with GPU Mich Talebzadeh
  • Re: Spark with GPU Jack Goodson
  • Re: Spark with GPU Alessandro Bellina
• [no subject] GAURAV GUPTA
• pyspark not starting Kelum Perera
• Joins internally Sid
• Memory leak while caching in foreachBatch block kineret M
• [Spark SQL] Omit Create Table Statement in Spark Sql 阿强
  • Re: [Spark SQL] Omit Create Table Statement in Spark Sql pengyh
• Spark program not receiving messages from Cloud Pubsub Pramod Biligiri
  • Re: Spark program not receiving messages from Cloud Pubsub Pramod Biligiri
  • High number of tasks when ran on a hybrid cluster murat migdisoglu
• Spark Scala API still not updated for 2.13 or it's a mistake? Roman I
  • Re: Spark Scala API still not updated for 2.13 or it's a mistake? Sean Owen
  • Re: Spark Scala API still not updated for 2.13 or it's a mistake? Roman I
  • Re: Spark Scala API still not updated for 2.13 or it's a mistake? Sean Owen
  • Re: Spark Scala API still not updated for 2.13 or it's a mistake? pengyh
• log transfering into hadoop/spark pengyh
  • Re: log transfering into hadoop/spark ayan guha
  • Re: log transfering into hadoop/spark Gourav Sengupta
• [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Kumba Janga
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Sean Owen
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Kumba Janga
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty ayan guha
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Sean Owen
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Stelios Philippou
  • Re: [pyspark delta] [delta][Spark SQL]: Getting an Analysis Exception. The associated location (path) is not empty Sean Owen
• WARN: netlib.BLAS 陈刚
  • Re: WARN: netlib.BLAS Sean Owen
• Use case idea Gioele Sal. Perri
  • Re: Use case idea pengyh
  • Re: Use case idea Gourav Sengupta
  • Re: Use case idea pengyh
  • Re: Use case idea Gourav Sengupta
  • Re: Use case idea pengyh
• Salting technique doubt Sid
  • Re: Salting technique doubt Amit Joshi
  • Re: Salting technique doubt Sid
  • Re: Salting technique doubt Amit Joshi
  • Re: Salting technique doubt Jacob Lynn
  • Re: Salting technique doubt ayan guha
  • Re: Salting technique doubt Vinod KC
  • Re: Salting technique doubt Sid
• [no subject] Milin Korath
• PySpark cores Andrew Melo
  • Re: PySpark cores Jacob Lynn
  • Re: PySpark cores Gourav Sengupta
• spark can't connect to kafka via sasl_ssl wilson
  • Re: spark can't connect to kafka via sasl_ssl wilson
• RE: Spark Avro Java 17 Compatibility Shivaraj Sivasankaran
  • Re: Spark Avro Java 17 Compatibility Sean Owen
• [Spark thread pool configurations]: I would like to configure all ThreadPoolExecutor parameters for each thread pool started in Spark Alex Peelman
• Spark SQL Query filter behavior with special characters prashanth reddy
• Partial data with ADLS Gen2 kineret M
  • Re: [EXTERNAL] Partial data with ADLS Gen2 Shay Elbaz
  • Re: [EXTERNAL] Partial data with ADLS Gen2 Tufan Rakshit
  • Re: [EXTERNAL] Partial data with ADLS Gen2 hwl17801341688
• Updating Broadcast Variable in Spark Streaming 2.4.4 Dipl.-Inf. Rico Bergmann
  • Re: Updating Broadcast Variable in Spark Streaming 2.4.4 Sean Owen
  • Updating Broadcast Variable in Spark Streaming 2.4.4 Dipl.-Inf. Rico Bergmann
  • Re: Updating Broadcast Variable in Spark Streaming 2.4.4 Sean Owen
• Spark Structured Streaming -- Cannot consume next messages KhajaAsmath Mohammed
  • Re: Spark Structured Streaming -- Cannot consume next messages Artemis User
  • Re: Spark Structured Streaming -- Cannot consume next messages KhajaAsmath Mohammed
• external table with parquet files: problem querying in sparksql since data is stored as integer while hive schema expects a timestamp Joris Billen
  • Re: external table with parquet files: problem querying in sparksql since data is stored as integer while hive schema expects a timestamp Gourav Sengupta
• Pyspark and multiprocessing Bjørn Jørgensen
  • Fwd: Pyspark and multiprocessing Bjørn Jørgensen
  • Re: Pyspark and multiprocessing Khalid Mammadov
  • Re: Pyspark and multiprocessing Bjørn Jørgensen
  • Re: Pyspark and multiprocessing Khalid Mammadov
• [MLlib] Differences after version upgrade Roger Wechsler
  • Re: [MLlib] Differences after version upgrade Sean Owen
• Dependencies issue in spark rajat kumar
  • Re: Dependencies issue in spark rajat kumar
• Building a ML pipeline with no training Edgar H
  • Re: Building a ML pipeline with no training Sean Owen
• spark.executor.pyspark.memory not added to the executor resource request on Kubernetes Shay Elbaz
  • Re: spark.executor.pyspark.memory not added to the executor resource request on Kubernetes Shay Elbaz
• very simple UI on webpage to display x/y plots+histogram of data stored in hive Joris Billen
  • Re: very simple UI on webpage to display x/y plots+histogram of data stored in hive Sean Owen
  • Re: very simple UI on webpage to display x/y plots+histogram of data stored in hive Joris Billen
  • Re: very simple UI on webpage to display x/y plots+histogram of data stored in hive ayan guha
• Issue while building spark project rajat kumar
  • Re: Issue while building spark project Sean Owen
  • Re: Issue while building spark project rajat kumar
• CVE-2022-33891: Apache Spark shell command injection vulnerability via Spark UI Sean Owen
• [ANNOUNCE] Apache Spark 3.2.2 released Dongjoon Hyun
• Question regarding how to make spar Scala to evenly divide the spark job between executors Orkhan Dadashov
  • Re: Question regarding how to make spar Scala to evenly divide the spark job between executors Tufan Rakshit
• spark re-use shuffle files not happening Koert Kuipers
  • Re: [EXTERNAL] spark re-use shuffle files not happening Shay Elbaz
  • Re: [EXTERNAL] spark re-use shuffle files not happening Koert Kuipers
• Spark Convert Column to String Gibson
• [Building] Building with JDK11 Szymon Kuryło
  • Re: [Building] Building with JDK11 Sean Owen
  • Re: [Building] Building with JDK11 Tufan Rakshit
  • Re: [Building] Building with JDK11 Stephen Coy
  • Re: [Building] Building with JDK11 Sergey B.
  • Re: [Building] Building with JDK11 Stephen Coy
  • Re: [Building] Building with JDK11 Szymon Kuryło
  • Re: [Building] Building with JDK11 Gera Shegalov
  • Re: [Building] Building with JDK11 Sean Owen
• Spark (K8S) IPv6 support Valer
  • Re: Spark (K8S) IPv6 support Sean Owen
• [Spark Structured Continous Processing] Plans for future left join support. Mikołaj Błaszczyk
• How use pattern matching in spark Sid
  • Re: How use pattern matching in spark Bjørn Jørgensen
• Spark streaming pending mircobatches queue max length Anil Dasari
  • Re: Spark streaming pending mircobatches queue max length Anil Dasari
• [Spark][Core] Resource Allocation Amin Borjian
  • Re: [Spark][Core] Resource Allocation Sungwoo Park
• about cpu cores Yong Walt
  • Re: about cpu cores Sean Owen
  • Re: about cpu cores Tufan Rakshit
  • Re: about cpu cores Yong Walt
  • Re: about cpu cores Tufan Rakshit
  • Re: about cpu cores Gourav Sengupta
• reading each JSON file from dataframe... Muthu Jayakumar
  • Re: reading each JSON file from dataframe... Enrico Minack
  • Re: reading each JSON file from dataframe... Muthu Jayakumar
  • Re: reading each JSON file from dataframe... Enrico Minack
  • Re: reading each JSON file from dataframe... ayan guha
  • Re: reading each JSON file from dataframe... Muthu Jayakumar
  • Re: reading each JSON file from dataframe... Gourav Sengupta
• RDD.pipe() for binary data Yuhao Zhang
  • Re: [EXTERNAL] RDD.pipe() for binary data Shay Elbaz
  • Re: [EXTERNAL] RDD.pipe() for binary data Yuhao Zhang
  • Re: [EXTERNAL] RDD.pipe() for binary data Sean Owen
  • Re: [EXTERNAL] RDD.pipe() for binary data Sebastian Piu
  • Re: [EXTERNAL] RDD.pipe() for binary data Andrew Melo
• Migration from Spark 2.4.0 to Spark 3.1.1 caused SortMergeJoin to change to BroadcastHashJoin igor cabral uchoa
  • Re: Migration from Spark 2.4.0 to Spark 3.1.1 caused SortMergeJoin to change to BroadcastHashJoin Tufan Rakshit
  • Re: Migration from Spark 2.4.0 to Spark 3.1.1 caused SortMergeJoin to change to BroadcastHashJoin igor cabral uchoa
• Reading parquet strips non-nullability from schema Greg Kopff
• Reading snappy/lz4 compressed csv/json files Yeachan Park
• Spark with Hive (Standalone) Metastore Ankur Khanna
  • Re: Spark with Hive (Standalone) Metastore Qian SUN

• Earlier messages
• Later messages