Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
I think it's better to reduce test matrix...
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Spark 2.2 has guava dependency problem and
[SPARK-21549](https://issues.apache.org/jira/browse/SPARK-21549).
I added a workaround for SPARK-21549.
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Oops, my travis result missed some results...
I'll fix this PR. This problem might come from HBase.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/436
Re-run Travis.
---
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio/pull/436
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/436
[PIO-125] Add Spark 2.2 support
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio spark22
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/23
Update project info
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio-sdk-python
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Thanks! I'll merge this PR tomorrow.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Added __init__.py. It will work on Python 2.7.
---
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
---
GitHub user marevol reopened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Add travis test and Refactoring
I'll merge this PR after
[PIO-117](https://issues.apache.org/jira/browse/PIO-117).
You can merge this pull request into a Git repository
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Run Travis.
---
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
Travis is not available in this repository...
I'll ask Infra to do that.
---
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/429
[PIO-118] ClassCastException from NullWritable to Text in ESEventsUtil
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/430
[PIO-119] Bump up Elasticsearch to 5.5.2
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio es552
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-sdk-python/pull/22
[WIP] Add travis test and Refactoring
I'll merge this PR after
[PIO-117](https://issues.apache.org/jira/browse/PIO-117).
You can merge this pull request into a Git
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/427
Thank you for checking it. Replaced with if-else.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/427
[PIO-116] PySpark Support
This PR provides PySpark support with minimum PIO changes.
1. Support pyspark on pio-shell
2. Add python files to use pyspark
3. Add --main
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/406
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/404
[PIO-100] Remove SLF4J warning message on pio command
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/398
[PIO-83][PIO-84] Move to scala 2.11.8, spark 2.1.1, elasticsearch 5.4.1 as
default env
You can merge this pull request into a Git repository by running:
$ git pull https
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/386
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/371
I would like to merge S3 support this week.
For travis testing, it passed in
https://travis-ci.org/jpioug/incubator-predictionio/builds/235569236 though
apache-travis was failed
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/371
Added LocalStack for S3 testing.
LocalStack/moto seems to have a problem about ETag handling...
I added DISABLE_CHUNKED_ENCODING option to avoid it in S3 testing
Github user marevol closed the pull request at:
https://github.com/apache/incubator-predictionio-template-recommender/pull/16
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113695041
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/374
ES storage improvement/refactoring
- Use the same ID generation for ES in event data
- Reuse RestClient instance in ESLEvents
- Replace 0 with false in mapping
- Set
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113576948
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113334667
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/373
Thank you for merging it.
Our production environment runs on ES 5.1, and my local dev is on ES 5.3.
---
If your project is set up for it, you can reply to this email and have
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/373
Improve insert performance for ES storage
- Change properties and n to not indexed field
- Set refresh to false (_version does not need to be refreshed)
and modified pio
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r112782773
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/371
[PIO-61] Add S3 Model Data Repository
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio storage_s3
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/370
Add EVENTDATA_REFRESH for elasticsearch
To improve indexing performance for event data, it's better to be able to
set refresh parameter.
In PIO, refresh is true by default
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/369
Add parameters to specify the number of shards/replicas for elasticsearch
To set the number of shards/replicas, add the following values:
- PIO_STORAGE_SOURCES_ELASTICSEARCH_
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
Synced up with develop branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/359
Using make-distribution.sh, I'll try to modify it to keep the directory
structure.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/359#discussion_r106052297
--- Diff: .travis.yml ---
@@ -47,7 +47,7 @@ env:
- BUILD_TYPE=Unit METADATA_REP=PGSQL EVENTDATA_REP=PGSQL
MODELDATA_REP=PGSQL
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/359
[PIO-57] Add SBT Native Packager
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio
sbt-native
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/358
Update event serialization for ES5
- Modify Event index mapping (change properties field to keyword type)
- Use UTC time format for eventTime/creatationTime
You can merge
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/347
@dszeto Could you check and merge this pull request?
He is my co-worker and Scala expert.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Fixed problems for ES5 support.
Integration tests will be passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Almost all failures are fixed.
For remaining problem, integration tests use PDT time, but elasticsearch
uses UTC time.
Therefore, the following assertion is failed in
tests
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
For integration tests of ES5, 3 tests are failed on
[move-storages-es5]...(https://github.com/marevol/incubator-predictionio/tree/move-storages-es5)
I'll check them today
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/355
Please see https://github.com/apache/incubator-predictionio/pull/352
We are working on Elasticsearch 5 support.
This fix removes elasticsearch from core.
---
If your project
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Correct.
I'm working on integration tests for ES5. To use ES5 as default, should I
create a new PR?
For UR, @haginot and co-workers work on ES5 support
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
Fixed!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/352
I'll fix it today...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Right. To use ES5, please replace pio-data-elasticsearch1-assembly-\*.jar
with pio-data-elasticsearch-assembly-\*.jar in lib/spark directory.
To change JARs location(ex. lib
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Please see https://github.com/apache/incubator-predictionio/pull/352
- Move storage implementation to storage directory
- Put storage assembly JARs into lib/spark
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/352
[PIO-49] Add support for Elasticsearch 5 and move storage implementation
You can merge this pull request into a Git repository by running:
$ git pull https://github.com
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
+1
As @pferrel 's comment, I think it may be better to use a directory(ex.
lib/spark) other than lib if we want to add arbitrary JARs to spark-submit.
If we put them into lib
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
compute-classpath.sh deals with plugins directory, but existing code seems
not to put jars in spark-submit.
If plugin feature looks good, I'll create PR for feature/es5
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Although I'm not sure about old plugin features, in my fix, I put plugin
jar files with --jars on spark-submit:
https://github.com/jpioug/incubator-predictionio/commit
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Please see our snapshot build:
http://fess.codelibs.org/snapshot/apache-predictionio-0.11.0-v1-SNAPSHOT.zip
Storage implementations, such as hbase, were moved to plugins/data
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio/pull/349
[PIO-54] Avoid access key started with -
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marevol/incubator-predictionio retry
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
@dszeto Could you replace ELASTICSEARCH1 with ELASTICSEARCH5?
https://github.com/apache/incubator-predictionio/blob/feature/es5/bin/pio-start-all#L37,L38
feature/es5
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
@dszeto Did you work on this issue? I think I'm available this week. So,
I'll do that if you do not work yet.
---
If your project is set up for it, you can reply to this email
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
To use the existing code for elasticsearch 1.x, please use
PIO_STORAGE_REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH1 and
PIO_STORAGE_SOURCES_ELASTICSEARCH1_* settings
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
Thank you for your comment.
We will check if it keeps the code for ES 1.x.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user marevol commented on the issue:
https://github.com/apache/incubator-predictionio/pull/336
We use the following template.
https://github.com/marevol/incubator-predictionio-template-recommender/tree/0.11.0
(Commented out "data.ratings.take(1)", becau
GitHub user marevol opened a pull request:
https://github.com/apache/incubator-predictionio-template-recommender/pull/12
Python 3 support for data/*.py
Adding () to print statement, it works on Python 3.5.2.
You can merge this pull request into a Git repository by running
68 matches
Mail list logo
