[jira] [Updated] (SPARK-14160) Windowing for structured streaming

[ https://issues.apache.org/jira/browse/SPARK-14160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-14160: Description: This JIRA is to track the status regarding event time windowing operations for

[jira] [Created] (SPARK-14160) Windowing for structured streaming

Burak Yavuz created SPARK-14160: --- Summary: Windowing for structured streaming Key: SPARK-14160 URL: https://issues.apache.org/jira/browse/SPARK-14160 Project: Spark Issue Type: Sub-task

Re: [VOTE] Release Apache Spark 1.6.1 (RC1)

+1 On Tue, Mar 8, 2016 at 10:59 AM, Andrew Or wrote: > +1 > > 2016-03-08 10:59 GMT-08:00 Yin Huai : > >> +1 >> >> On Mon, Mar 7, 2016 at 12:39 PM, Reynold Xin wrote: >> >>> +1 (binding) >>> >>> >>> On Sun, Mar 6, 2016 at 12:08 PM, Egor Pahomov >>> wrote: >>> +1 Spark ODBC server

Re: RethinkDB as a Datasource

Hi, You can always write it as a data source and share it on Spark Packages. There are many data source connectors available already: http://spark-packages.org/?q=tags%3A%22Data%20Sources%22 Best, Burak On Fri, Mar 4, 2016 at 3:19 PM, pnakibar wrote: > Hi, > I see that there is no way to use R

Re: Calculation of histogram bins and frequency in Apache spark 1.6

You could use the Bucketizer transformer in Spark ML. Best, Burak On Tue, Feb 23, 2016 at 9:13 AM, Arunkumar Pillai wrote: > Hi > Is there any predefined method to calculate histogram bins and frequency > in spark. Currently I take range and find bins then count frequency using > SQL query. > >

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] ef34c8: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: ef34c811ad9b74897382c4f38a11bbfc3350d7d3 https://github.com/phpmyadmin/phpmyadmin/commit/ef34c811ad9b74897382c4f38a11bbfc3350d7d3 Author: Burak Yavuz Date: 2016-02-22 (Mon, 22 Feb 2016

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 82190f: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_6 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 82190f978a905c0227514a4c04d2356cd293573a https://github.com/phpmyadmin/phpmyadmin/commit/82190f978a905c0227514a4c04d2356cd293573a Author: Burak Yavuz Date: 2016-02-17 (Wed, 17 Feb 2016

Re: Using SPARK packages in Spark Cluster

Hello Gourav, The packages need to be loaded BEFORE you start the JVM, therefore you won't be able to add packages dynamically in code. You should use the --packages with pyspark before you start your application. One option is to add a `conf` that will load some packages if you are constantly goi

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 97a9a8: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_6 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 97a9a8bbd8867104ad9b09114d681cf4f1f869bb https://github.com/phpmyadmin/phpmyadmin/commit/97a9a8bbd8867104ad9b09114d681cf4f1f869bb Author: Burak Yavuz Date: 2016-02-08 (Mon, 08 Feb 2016

Re: Guidelines for writing SPARK packages

Thanks for the reply David, just wanted to fix one part of your response: > If you > want to register a release for your package you will also need to push > the artifacts for your package to Maven central. > It is NOT necessary to push to Maven Central in order to make a release. There are many

Re: Redirect Spark Logs to Kafka

You can use the KafkaLog4jAppender ( https://github.com/apache/kafka/blob/trunk/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java ). Best, Burak On Mon, Feb 1, 2016 at 12:20 PM, Ashish Soni wrote: > Hi All , > > Please let me know how we can redirect spark logg

Re: Spark not able to fetch events from Amazon Kinesis

Hi Yash, I've run into multiple problems due to version incompatibilities, either due to protobuf or jackson. That may be your culprit. The problem is that all failures by the Kinesis Client Lib is silent, therefore don't show up on the logs. It's very hard to debug those buggers. Best, Burak On

Re: Optimized way to multiply two large matrices and save output using Spark and Scala

BlockMatrix.multiply is the suggested method of multiplying two large matrices. Is there a reason that you didn't use BlockMatrices? You can load the matrices and convert to and from RowMatrix. If it's in sparse format (i, j, v), then you can also use the CoordinateMatrix to load, BlockMatrix to m

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] bd7f28: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: bd7f28f8f794dad75c07221c31795200c65e2d30 https://github.com/phpmyadmin/phpmyadmin/commit/bd7f28f8f794dad75c07221c31795200c65e2d30 Author: Burak Yavuz Date: 2016-01-08 (Fri, 08 Jan 2016

[phpMyAdmin Git] [phpmyadmin/localized_docs] c40985: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: c409855781c71d37747a410f32f3c8d162c62a95 https://github.com/phpmyadmin/localized_docs/commit/c409855781c71d37747a410f32f3c8d162c62a95 Author: Burak Yavuz Date: 2015-12-29 (Tue, 29 Dec 2015

[phpMyAdmin Git] [phpmyadmin/localized_docs] 5c6a97: Translated using Weblate (German)

/ed6b1323a64a99ec31a1950dbe9433bfbc6dd311 Author: Burak Yavuz Date: 2015-12-28 (Mon, 28 Dec 2015) Changed paths: M po/tr.mo M po/tr.po Log Message: --- Translated using Weblate (Turkish) Currently translated at 99.2% (1918 of 1933 strings) [CI skip] Compare: https

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 27be93: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 27be93ab6d19cc0afb24b212e6e7ceee4ea317db https://github.com/phpmyadmin/phpmyadmin/commit/27be93ab6d19cc0afb24b212e6e7ceee4ea317db Author: Burak Yavuz Date: 2015-12-27 (Sun, 27 Dec 2015

Re: number of blocks in ALS/recommendation API

Copying the first part from the scaladoc: " This is a blocked implementation of the ALS factorization algorithm that groups the two sets of factors (referred to as "users" and "products") into blocks and reduces communication by only sending one copy of each user vector to each product block on eac

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] f39e51: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: f39e5147b49e03097e954d1dc344659d56536808 https://github.com/phpmyadmin/phpmyadmin/commit/f39e5147b49e03097e954d1dc344659d56536808 Author: Burak Yavuz Date: 2015-12-17 (Thu, 17 Dec 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] e6ca04: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: e6ca04f83aae09b7dbc187b0fdaaca85b7e8f1f1 https://github.com/phpmyadmin/phpmyadmin/commit/e6ca04f83aae09b7dbc187b0fdaaca85b7e8f1f1 Author: Burak Yavuz Date: 2015-12-16 (Wed, 16 Dec 2015

Re: [VOTE] Release Apache Spark 1.6.0 (RC2)

+1 tested SparkSQL and Streaming on some production sized workloads On Sat, Dec 12, 2015 at 4:16 PM, Mark Hamstra wrote: > +1 > > On Sat, Dec 12, 2015 at 9:39 AM, Michael Armbrust > wrote: > >> Please vote on releasing the following candidate as Apache Spark version >> 1.6.0! >> >> The vote is

Re: Spark streaming with Kinesis broken?

6 branch as of commit >> db5165246f2888537dd0f3d4c5a515875c7358ed. That makes it much less >> serious of an issue, although it would be nice to know what the root cause >> is to avoid a regression. >> >> On Thu, Dec 10, 2015 at 4:03 PM Burak Yavuz wrote: >> >>

Re: Spark streaming with Kinesis broken?

I've noticed this happening when there was some dependency conflicts, and it is super hard to debug. It seems that the KinesisClientLibrary version in Spark 1.5.2 is 1.3.0, but it is 1.2.1 in Spark 1.5.1. I feel like that seems to be the problem... Brian, did you verify that it works with the 1.6.

[jira] [Updated] (SPARK-12106) Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

[ https://issues.apache.org/jira/browse/SPARK-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-12106: Description: This test is still transiently flaky, because async methods can finish out of order

[jira] [Created] (SPARK-12106) Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

Burak Yavuz created SPARK-12106: --- Summary: Flaky Test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry Key: SPARK-12106 URL: https://issues.apache.org/jira/browse/SPARK

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 502d72: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 502d728c8fa9bc682b3dba15580ffc4df767dd1b https://github.com/phpmyadmin/phpmyadmin/commit/502d728c8fa9bc682b3dba15580ffc4df767dd1b Author: Burak Yavuz Date: 2015-12-02 (Wed, 02 Dec 2015

Re: Export BLAS module on Spark MLlib

Or you could also use reflection like in this Spark Package: https://github.com/brkyvz/lazy-linalg/blob/master/src/main/scala/com/brkyvz/spark/linalg/BLASUtils.scala Best, Burak On Mon, Nov 30, 2015 at 12:48 PM, DB Tsai wrote: > The workaround is have your code in the same package, or write som

[jira] [Created] (SPARK-11985) Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler

Burak Yavuz created SPARK-11985: --- Summary: Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler Key: SPARK-11985 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Updated] (SPARK-11985) Update Spark Streaming - Kinesis Library Documentation regarding data de-aggregation and message handler

[ https://issues.apache.org/jira/browse/SPARK-11985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-11985: Description: Update documentation and provide how-to example in guide. > Update Spark Stream

Re: Spark Streaming idempotent writes to HDFS

Not sure if it would be the most efficient, but maybe you can think of the filesystem as a key value store, and write each batch to a sub-directory, where the directory name is the batch time. If the directory already exists, then you shouldn't write it. Then you may have a following batch job that

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 7d4aa6: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 7d4aa69470082c559894fe89f47d594071f30f77 https://github.com/phpmyadmin/phpmyadmin/commit/7d4aa69470082c559894fe89f47d594071f30f77 Author: Burak Yavuz Date: 2015-11-18 (Wed, 18 Nov 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] a039b6: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: a039b6232ae3f45e0115411587f088b419cb3b63 https://github.com/phpmyadmin/phpmyadmin/commit/a039b6232ae3f45e0115411587f088b419cb3b63 Author: Burak Yavuz Date: 2015-11-17 (Tue, 17 Nov 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] c7378b: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: c7378b15cff94f1780969ce4c181d7a5a1c00f2c https://github.com/phpmyadmin/phpmyadmin/commit/c7378b15cff94f1780969ce4c181d7a5a1c00f2c Author: Burak Yavuz Date: 2015-11-14 (Sat, 14 Nov 2015

Re: large, dense matrix multiplication

Hi, The BlockMatrix multiplication should be much more efficient on the current master (and will be available with Spark 1.6). Could you please give that a try if you have the chance? Thanks, Burak On Fri, Nov 13, 2015 at 10:11 AM, Sabarish Sasidharan < sabarish.sasidha...@manthan.com> wrote: >

[jira] [Created] (SPARK-11731) Enable batching on Driver WriteAheadLog by default

Burak Yavuz created SPARK-11731: --- Summary: Enable batching on Driver WriteAheadLog by default Key: SPARK-11731 URL: https://issues.apache.org/jira/browse/SPARK-11731 Project: Spark Issue Type

[phpMyAdmin Git] [phpmyadmin/localized_docs] abb554: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: abb554fa28ccaee33437b0b2d2b1da52a81b912e https://github.com/phpmyadmin/localized_docs/commit/abb554fa28ccaee33437b0b2d2b1da52a81b912e Author: Burak Yavuz Date: 2015-11-13 (Fri, 13 Nov 2015

[phpMyAdmin Git] [phpmyadmin/localized_docs] d18291: Translated using Weblate (Albanian)

translated at 87.2% (1667 of 1911 strings) [CI skip] Commit: 72697ec297bb8c4f31ac3db3df37117aa2feaaeb https://github.com/phpmyadmin/localized_docs/commit/72697ec297bb8c4f31ac3db3df37117aa2feaaeb Author: Burak Yavuz Date: 2015-11-12 (Thu, 12 Nov 2015) Changed paths: M po/tr.mo

Re: Spark Packages Configuration Not Found

Hi Jakob, > As another, general question, are spark packages the go-to way of extending spark functionality? Definitely. There are ~150 Spark Packages out there in spark-packages.org. I use a lot of them in every day Spark work. The number of released packages have steadily increased rate over the

[jira] [Created] (SPARK-11639) Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry

Burak Yavuz created SPARK-11639: --- Summary: Flaky test: BatchedWriteAheadLog - name log with aggregated entries with the timestamp of last entry Key: SPARK-11639 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Commented] (SPARK-11198) Support record de-aggregation in KinesisReceiver

[ https://issues.apache.org/jira/browse/SPARK-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985701#comment-14985701 ] Burak Yavuz commented on SPARK-11198: - Just tested this. It works during reg

[jira] [Commented] (SPARK-11198) Support record de-aggregation in KinesisReceiver

[ https://issues.apache.org/jira/browse/SPARK-11198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982926#comment-14982926 ] Burak Yavuz commented on SPARK-11198: - [~boneill42], did you need to do anyt

[jira] [Created] (SPARK-11419) WriteAheadLog recovery improvements for when closeFileAfterWrite is enabled

Burak Yavuz created SPARK-11419: --- Summary: WriteAheadLog recovery improvements for when closeFileAfterWrite is enabled Key: SPARK-11419 URL: https://issues.apache.org/jira/browse/SPARK-11419 Project

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] a63075: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: a63075fed9af3cb89cbe889f3103bfc8338185e8 https://github.com/phpmyadmin/phpmyadmin/commit/a63075fed9af3cb89cbe889f3103bfc8338185e8 Author: Burak Yavuz Date: 2015-10-27 (Tue, 27 Oct 2015

[jira] [Created] (SPARK-11324) Flag to close Write Ahead Log after writing

Burak Yavuz created SPARK-11324: --- Summary: Flag to close Write Ahead Log after writing Key: SPARK-11324 URL: https://issues.apache.org/jira/browse/SPARK-11324 Project: Spark Issue Type

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 2a1db0: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 2a1db0a27fcfdfe90e496a8a316642c404f620bf https://github.com/phpmyadmin/phpmyadmin/commit/2a1db0a27fcfdfe90e496a8a316642c404f620bf Author: Burak Yavuz Date: 2015-10-23 (Fri, 23 Oct 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 6b5b3b: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 6b5b3b395bb73ff3e6bb1948b1c69d379b78ae7d https://github.com/phpmyadmin/phpmyadmin/commit/6b5b3b395bb73ff3e6bb1948b1c69d379b78ae7d Author: Burak Yavuz Date: 2015-10-23 (Fri, 23 Oct 2015

[phpMyAdmin Git] [phpmyadmin/localized_docs] 330452: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: 330452e6e5f8f9107ab8ceb0070f254b91a1f30a https://github.com/phpmyadmin/localized_docs/commit/330452e6e5f8f9107ab8ceb0070f254b91a1f30a Author: Burak Yavuz Date: 2015-10-16 (Fri, 16 Oct 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] a2f0f1: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: a2f0f17cab64d953172fed279cf76a104f5f99bd https://github.com/phpmyadmin/phpmyadmin/commit/a2f0f17cab64d953172fed279cf76a104f5f99bd Author: Burak Yavuz Date: 2015-10-16 (Fri, 16 Oct 2015

[jira] [Created] (SPARK-11141) Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes

Burak Yavuz created SPARK-11141: --- Summary: Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes Key: SPARK-11141 URL: https://issues.apache.org/jira/browse/SPARK-11141 Project: Spark

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 5a4022: Translated using Weblate (Finnish)

strings) [CI skip] Commit: 7aa6cf23ea5035be48a1b8c03548882cd363afcb https://github.com/phpmyadmin/phpmyadmin/commit/7aa6cf23ea5035be48a1b8c03548882cd363afcb Author: Burak Yavuz Date: 2015-10-14 (Wed, 14 Oct 2015) Changed paths: M po/tr.po Log Message

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 0ee2e5: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 0ee2e5060e8342fed72fbd7cd52b7403aac3ad4b https://github.com/phpmyadmin/phpmyadmin/commit/0ee2e5060e8342fed72fbd7cd52b7403aac3ad4b Author: Burak Yavuz Date: 2015-10-13 (Tue, 13 Oct 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] a1185d: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: a1185d27693dcd2f2a23d7ff4b144c2002d6cd8a https://github.com/phpmyadmin/phpmyadmin/commit/a1185d27693dcd2f2a23d7ff4b144c2002d6cd8a Author: Burak Yavuz Date: 2015-10-09 (Fri, 09 Oct 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] fa58a9: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: fa58a99ad2f48e09f5bafb440c70b822a6e29de0 https://github.com/phpmyadmin/phpmyadmin/commit/fa58a99ad2f48e09f5bafb440c70b822a6e29de0 Author: Burak Yavuz Date: 2015-10-09 (Fri, 09 Oct 2015

Re: spark-submit --packages using different resolver

Hi Jerry, The --packages feature doesn't support private repositories right now. However, in the case of s3, maybe it might work. Could you please try using the --repositories flag and provide the address: `$ spark-submit --packages my:awesome:package --repositories s3n://$aws_ak:$aws_sak@bucket/p

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 41a595: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 41a59578c193dcbfb6548644bb58e7de4cb27427 https://github.com/phpmyadmin/phpmyadmin/commit/41a59578c193dcbfb6548644bb58e7de4cb27427 Author: Burak Yavuz Date: 2015-10-02 (Fri, 02 Oct 2015

[jira] [Created] (SPARK-10891) Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka

Burak Yavuz created SPARK-10891: --- Summary: Add MessageHandler to KinesisUtils.createStream similar to Direct Kafka Key: SPARK-10891 URL: https://issues.apache.org/jira/browse/SPARK-10891 Project: Spark

[jira] [Commented] (SPARK-10889) Upgrade Kinesis Client Library

[ https://issues.apache.org/jira/browse/SPARK-10889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939023#comment-14939023 ] Burak Yavuz commented on SPARK-10889: - In addition, KCL 1.4.0 support

[phpMyAdmin Git] [phpmyadmin/localized_docs] bb447e: Translated using Weblate (Armenian)

(Slovenian) Currently translated at 6.7% (116 of 1724 strings) [CI skip] Commit: e9ee84ea686d81b08bac7cb1b4e0622b17630752 https://github.com/phpmyadmin/localized_docs/commit/e9ee84ea686d81b08bac7cb1b4e0622b17630752 Author: Burak Yavuz Date: 2015-09-30 (Wed, 30 Sep 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 7d4e44: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 7d4e445a300c02cb128ff6c302dbd1366859131d https://github.com/phpmyadmin/phpmyadmin/commit/7d4e445a300c02cb128ff6c302dbd1366859131d Author: Burak Yavuz Date: 2015-09-29 (Tue, 29 Sep 2015

[jira] [Updated] (SPARK-10599) Decrease communication in BlockMatrix multiply and increase performance

[ https://issues.apache.org/jira/browse/SPARK-10599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-10599: Description: The BlockMatrix multiply sends each block to all the corresponding columns of the

[jira] [Created] (SPARK-10599) Decrease communication in BlockMatrix multiply and increase performance

Burak Yavuz created SPARK-10599: --- Summary: Decrease communication in BlockMatrix multiply and increase performance Key: SPARK-10599 URL: https://issues.apache.org/jira/browse/SPARK-10599 Project: Spark

[phpMyAdmin Git] [phpmyadmin/localized_docs] e5ad8a: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: e5ad8aeeb0b9d45d2aee9f74cdc645e3e262369b https://github.com/phpmyadmin/localized_docs/commit/e5ad8aeeb0b9d45d2aee9f74cdc645e3e262369b Author: Burak Yavuz Date: 2015-09-12 (Sat, 12 Sep 2015

Re: Adding/subtracting org.apache.spark.mllib.linalg.Vector in Scala?

On Tue, Aug 25, 2015 at 1:57 PM, Burak Yavuz wrote: > >> Hmm. I have a lot of code on the local linear algebra operations using >> Spark's Matrix and Vector representations >> done for https://issues.apache.org/jira/browse/SPARK-6442. >> >> I can make a

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 057263: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 05726349f0a123ca62b2a599960445c26b63d29b https://github.com/phpmyadmin/phpmyadmin/commit/05726349f0a123ca62b2a599960445c26b63d29b Author: Burak Yavuz Date: 2015-09-06 (Sun, 06 Sep 2015

Re: [VOTE] Release Apache Spark 1.5.0 (RC3)

+1. Tested complex R package support (Scala + R code), BLAS and DataFrame fixes good. Burak On Thu, Sep 3, 2015 at 8:56 AM, mkhaitman wrote: > Built and tested on CentOS 7, Hadoop 2.7.1 (Built for 2.6 profile), > Standalone without any problems. Re-tested dynamic allocation specifically. > > "L

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 02f708: Translated using Weblate (Albanian)

3209 strings) [CI skip] Commit: ed21004fdc9a62ca57955780c00fefc198768cde https://github.com/phpmyadmin/phpmyadmin/commit/ed21004fdc9a62ca57955780c00fefc198768cde Author: Burak Yavuz Date: 2015-09-01 (Tue, 01 Sep 2015) Changed paths: M po/tr.po Log Message

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 9656d6: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_5 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 9656d600b992c275a7179414a70990e63a828823 https://github.com/phpmyadmin/phpmyadmin/commit/9656d600b992c275a7179414a70990e63a828823 Author: Burak Yavuz Date: 2015-08-31 (Mon, 31 Aug 2015

[jira] [Updated] (SPARK-10353) MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication

[ https://issues.apache.org/jira/browse/SPARK-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz updated SPARK-10353: Affects Version/s: 1.5.0 > MLlib BLAS gemm outputs wrong result when beta = 0.0 for transp

[jira] [Created] (SPARK-10353) MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication

Burak Yavuz created SPARK-10353: --- Summary: MLlib BLAS gemm outputs wrong result when beta = 0.0 for transpose transpose matrix multiplication Key: SPARK-10353 URL: https://issues.apache.org/jira/browse/SPARK-10353

Re: Calculating Min and Max Values using Spark Transformations?

Or you can just call describe() on the dataframe? In addition to min-max, you'll also get the mean, and count of non-null and non-NA elements as well. Burak On Fri, Aug 28, 2015 at 10:09 AM, java8964 wrote: > Or RDD.max() and RDD.min() won't work for you? > > Yong > > --

[phpMyAdmin Git] [phpmyadmin/localized_docs] ee9be6: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: ee9be6b3473bb3b71f639bfb36be1dba2e55c0a2 https://github.com/phpmyadmin/localized_docs/commit/ee9be6b3473bb3b71f639bfb36be1dba2e55c0a2 Author: Burak Yavuz Date: 2015-08-27 (Thu, 27 Aug 2015

[phpMyAdmin Git] [phpmyadmin/localized_docs] 2bf838: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: 2bf8388b70b0578132bc9f62b80da70c57ea1c5f https://github.com/phpmyadmin/localized_docs/commit/2bf8388b70b0578132bc9f62b80da70c57ea1c5f Author: Burak Yavuz Date: 2015-08-26 (Wed, 26 Aug 2015

[phpMyAdmin Git] [phpmyadmin/localized_docs] e6440d: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: e6440d74dd9e8d50d45de6055a984de582d29684 https://github.com/phpmyadmin/localized_docs/commit/e6440d74dd9e8d50d45de6055a984de582d29684 Author: Burak Yavuz Date: 2015-08-26 (Wed, 26 Aug 2015

Re: Adding/subtracting org.apache.spark.mllib.linalg.Vector in Scala?

Hmm. I have a lot of code on the local linear algebra operations using Spark's Matrix and Vector representations done for https://issues.apache.org/jira/browse/SPARK-6442. I can make a Spark package with that code if people are interested. Best, Burak On Tue, Aug 25, 2015 at 10:54 AM, Kristina R

Re: Unable to catch SparkContext methods exceptions

> > Best, > Roberto > > > On Mon, Aug 24, 2015 at 7:38 PM, Burak Yavuz wrote: > >> textFile is a lazy operation. It doesn't evaluate until you call an >> action on it, such as .count(). Therefore, you won't catch the exception >> there. >>

Re: Unable to catch SparkContext methods exceptions

textFile is a lazy operation. It doesn't evaluate until you call an action on it, such as .count(). Therefore, you won't catch the exception there. Best, Burak On Mon, Aug 24, 2015 at 9:09 AM, Roberto Coluccio < roberto.coluc...@gmail.com> wrote: > Hello folks, > > I'm experiencing an unexpected

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 70b5c9: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 70b5c9ddeb96b00090e24de363464b3058dffa41 https://github.com/phpmyadmin/phpmyadmin/commit/70b5c9ddeb96b00090e24de363464b3058dffa41 Author: Burak Yavuz Date: 2015-08-22 (Sat, 22 Aug 2015

Re: Convert mllib.linalg.Matrix to Breeze

Thursday 20 August 2015 07:50 PM, Burak Yavuz wrote: > > Matrix.toBreeze is a private method. MLlib matrices have the same > structure as Breeze Matrices. Just create a new Breeze matrix like this > <https://github.com/apache/spark/blob/43e0135421b2262cbb0e06aae53523f663b4f959/ml

Re: Creating Spark DataFrame from large pandas DataFrame

If you would like to try using spark-csv, please use `pyspark --packages com.databricks:spark-csv_2.11:1.2.0` You're missing a dependency. Best, Burak On Thu, Aug 20, 2015 at 1:08 PM, Charlie Hack wrote: > Hi, > > I'm new to spark and am trying to create a Spark df from a pandas df with > ~5 m

Re: Convert mllib.linalg.Matrix to Breeze

Matrix.toBreeze is a private method. MLlib matrices have the same structure as Breeze Matrices. Just create a new Breeze matrix like this . Best, B

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] eb4b3a: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: eb4b3a9ce6d83d3a990b0a8cf0def9d92de0af1e https://github.com/phpmyadmin/phpmyadmin/commit/eb4b3a9ce6d83d3a990b0a8cf0def9d92de0af1e Author: Burak Yavuz Date: 2015-08-15 (Sat, 15 Aug 2015

[jira] [Commented] (SPARK-10000) holder3

[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697689#comment-14697689 ] Burak Yavuz commented on SPARK-1: - I don't get what this issue is :S H

Re: Unit Testing

I would recommend this spark package for your unit testing needs ( http://spark-packages.org/package/holdenk/spark-testing-base). Best, Burak On Thu, Aug 13, 2015 at 5:51 AM, jay vyas wrote: > yes there certainly is, so long as eclipse has the right plugins and so on > to run scala programs. Y

[jira] [Created] (SPARK-9916) Clear leftover sparkr.zip copies and creations (e.g. make-distribution.sh)

Burak Yavuz created SPARK-9916: -- Summary: Clear leftover sparkr.zip copies and creations (e.g. make-distribution.sh) Key: SPARK-9916 URL: https://issues.apache.org/jira/browse/SPARK-9916 Project: Spark

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] e13e30: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: e13e30e172783ccc98734acc1b03afdd5e5aa293 https://github.com/phpmyadmin/phpmyadmin/commit/e13e30e172783ccc98734acc1b03afdd5e5aa293 Author: Burak Yavuz Date: 2015-08-12 (Wed, 12 Aug 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 88010e: Translated using Weblate (Interlingua)

https://github.com/phpmyadmin/phpmyadmin/commit/cd80cb61a09618d14a60f6d2f719494924624190 Author: Burak Yavuz Date: 2015-08-09 (Sun, 09 Aug 2015) Changed paths: M po/tr.po Log Message: --- Translated using Weblate (Turkish) Currently translated at 100.0% (3201 of 3201

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 7d853c: Translated using Weblate (Turkish)

Branch: refs/heads/QA_4_4 Home: https://github.com/phpmyadmin/phpmyadmin Commit: 7d853ce0edaa32b1cce55ec57d594a5398c2fd1c https://github.com/phpmyadmin/phpmyadmin/commit/7d853ce0edaa32b1cce55ec57d594a5398c2fd1c Author: Burak Yavuz Date: 2015-08-08 (Sat, 08 Aug 2015

[jira] [Commented] (SPARK-9742) NullPointerException when using --packages

[ https://issues.apache.org/jira/browse/SPARK-9742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662260#comment-14662260 ] Burak Yavuz commented on SPARK-9742: Did the behavior of Option's change

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661309#comment-14661309 ] Burak Yavuz commented on SPARK-9614: It used to work in Spark 1.4, without Tung

[phpMyAdmin Git] [phpmyadmin/localized_docs] 9f4c21: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/localized_docs Commit: 9f4c2132bf32cd23cbc6687b976ca101c81154a9 https://github.com/phpmyadmin/localized_docs/commit/9f4c2132bf32cd23cbc6687b976ca101c81154a9 Author: Burak Yavuz Date: 2015-08-05 (Wed, 05 Aug 2015

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] 09a0f3: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: 09a0f382a690085ed3ff471c88280a2fadcddbf7 https://github.com/phpmyadmin/phpmyadmin/commit/09a0f382a690085ed3ff471c88280a2fadcddbf7 Author: Burak Yavuz Date: 2015-08-05 (Wed, 05 Aug 2015

[jira] [Created] (SPARK-9616) Erroneous result in Frequent Items (SQL) when merging FrequentItemCounters

Burak Yavuz created SPARK-9616: -- Summary: Erroneous result in Frequent Items (SQL) when merging FrequentItemCounters Key: SPARK-9616 URL: https://issues.apache.org/jira/browse/SPARK-9616 Project: Spark

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654402#comment-14654402 ] Burak Yavuz commented on SPARK-9614: cc [~joshrosen] > InternalRow represe

[jira] [Created] (SPARK-9615) Use rdd.aggregate in FrequentItems

Burak Yavuz created SPARK-9615: -- Summary: Use rdd.aggregate in FrequentItems Key: SPARK-9615 URL: https://issues.apache.org/jira/browse/SPARK-9615 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

Burak Yavuz created SPARK-9614: -- Summary: InternalRow representation during executionPlan.toRdd.aggregete possibly problematic Key: SPARK-9614 URL: https://issues.apache.org/jira/browse/SPARK-9614

[jira] [Created] (SPARK-9603) Re-enable complex R package test in SparkSubmitSuite

Burak Yavuz created SPARK-9603: -- Summary: Re-enable complex R package test in SparkSubmitSuite Key: SPARK-9603 URL: https://issues.apache.org/jira/browse/SPARK-9603 Project: Spark Issue Type

Re: Cannot Import Package (spark-csv)

In addition, you do not need to use --jars with --packages. --packages will get the jar for you. Best, Burak On Mon, Aug 3, 2015 at 9:01 AM, Burak Yavuz wrote: > Hi, there was this issue for Scala 2.11. > https://issues.apache.org/jira/browse/SPARK-7944 > It should be fixed on mast

Re: Cannot Import Package (spark-csv)

Hi, there was this issue for Scala 2.11. https://issues.apache.org/jira/browse/SPARK-7944 It should be fixed on master branch. You may be hitting that. Best, Burak On Sun, Aug 2, 2015 at 9:06 PM, Ted Yu wrote: > I tried the following command on master branch: > bin/spark-shell --packages com.da

[phpMyAdmin Git] [phpmyadmin/phpmyadmin] ddd26c: Translated using Weblate (Turkish)

Branch: refs/heads/master Home: https://github.com/phpmyadmin/phpmyadmin Commit: ddd26c1e6e7cb2bbf8ecb42a01b029fdf717d2cf https://github.com/phpmyadmin/phpmyadmin/commit/ddd26c1e6e7cb2bbf8ecb42a01b029fdf717d2cf Author: Burak Yavuz Date: 2015-08-03 (Mon, 03 Aug 2015

Re: FrequentItems in spark-sql-execution-stat

Hi Yucheng, Thanks for pointing out the issue. You are correct, in the case that the final map is completely empty after the merge, we do need to add the final element to the map, with the correct count (decrement the count with the max count that was already in the map). I'll submit a fix for it.

<    2   3   4   5   6   7   8   9   10   11   >