[GitHub] incubator-predictionio pull request #373: Improve insert performance for ES ...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-predictionio/pull/373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-predictionio issue #373: Improve insert performance for ES storage
Github user dszeto commented on the issue: https://github.com/apache/incubator-predictionio/pull/373 LGTM. By the way, are you working with ES 5.2/5.3? @mars bumped into issues with ES 5.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-predictionio pull request #371: [PIO-61] Add S3 Model Data Reposit...
Github user marevol commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113334667
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import PIOBuild._
+
+name := "apache-predictionio-data-s3"
+
+libraryDependencies ++= Seq(
+ "org.apache.predictionio" %% "apache-predictionio-core" % version.value
% "provided",
+ "com.google.guava"% "guava" % "14.0.1"
% "provided",
+ "com.amazonaws" % "aws-java-sdk-s3" % "1.11.118",
+ "org.scalatest" %% "scalatest"% "2.1.7" %
"test")
+
+parallelExecution in Test := false
+
+pomExtra := childrenPomExtra.value
+
+assemblyOption in assembly := (assemblyOption in
assembly).value.copy(includeScala = false)
+
+assemblyShadeRules in assembly := Seq(
+ ShadeRule.rename("org.apache.http.**" ->
"shadeio.data.s3.http.@1").inAll,
+ ShadeRule.rename("com.fasterxml.**" ->
"shadeio.data.s3.fasterxml.@1").inAll
+)
--- End diff --
@mars could you provide info?
ES storage also conflicts with httpclient jar file of Spark.
I think an exception occurs if removing it, so it will be needed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-predictionio issue #373: Improve insert performance for ES storage
Github user marevol commented on the issue: https://github.com/apache/incubator-predictionio/pull/373 Thank you for merging it. Our production environment runs on ES 5.1, and my local dev is on ES 5.3. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-predictionio pull request #371: [PIO-61] Add S3 Model Data Reposit...
Github user dszeto commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113326594
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import PIOBuild._
+
+name := "apache-predictionio-data-s3"
+
+libraryDependencies ++= Seq(
+ "org.apache.predictionio" %% "apache-predictionio-core" % version.value
% "provided",
+ "com.google.guava"% "guava" % "14.0.1"
% "provided",
+ "com.amazonaws" % "aws-java-sdk-s3" % "1.11.118",
+ "org.scalatest" %% "scalatest"% "2.1.7" %
"test")
+
+parallelExecution in Test := false
+
+pomExtra := childrenPomExtra.value
+
+assemblyOption in assembly := (assemblyOption in
assembly).value.copy(includeScala = false)
+
+assemblyShadeRules in assembly := Seq(
+ ShadeRule.rename("org.apache.http.**" ->
"shadeio.data.s3.http.@1").inAll,
+ ShadeRule.rename("com.fasterxml.**" ->
"shadeio.data.s3.fasterxml.@1").inAll
+)
--- End diff --
I think @mars is seeing some issues with
https://github.com/apache/incubator-predictionio/blob/develop/storage/elasticsearch/build.sbt#L42.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (PIO-61) Add S3 Model Data Repository
[
https://issues.apache.org/jira/browse/PIO-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983742#comment-15983742
]
ASF GitHub Bot commented on PIO-61:
---
Github user dszeto commented on a diff in the pull request:
https://github.com/apache/incubator-predictionio/pull/371#discussion_r113326594
--- Diff: storage/s3/build.sbt ---
@@ -0,0 +1,44 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import PIOBuild._
+
+name := "apache-predictionio-data-s3"
+
+libraryDependencies ++= Seq(
+ "org.apache.predictionio" %% "apache-predictionio-core" % version.value
% "provided",
+ "com.google.guava"% "guava" % "14.0.1"
% "provided",
+ "com.amazonaws" % "aws-java-sdk-s3" % "1.11.118",
+ "org.scalatest" %% "scalatest"% "2.1.7" %
"test")
+
+parallelExecution in Test := false
+
+pomExtra := childrenPomExtra.value
+
+assemblyOption in assembly := (assemblyOption in
assembly).value.copy(includeScala = false)
+
+assemblyShadeRules in assembly := Seq(
+ ShadeRule.rename("org.apache.http.**" ->
"shadeio.data.s3.http.@1").inAll,
+ ShadeRule.rename("com.fasterxml.**" ->
"shadeio.data.s3.fasterxml.@1").inAll
+)
--- End diff --
I think @mars is seeing some issues with
https://github.com/apache/incubator-predictionio/blob/develop/storage/elasticsearch/build.sbt#L42.
> Add S3 Model Data Repository
>
>
> Key: PIO-61
> URL: https://issues.apache.org/jira/browse/PIO-61
> Project: PredictionIO
> Issue Type: Improvement
>Reporter: Shinsuke Sugaya
>
> To put/get model data to AWS S3 directly, using aws-java-sdk-s3, add
> storage/s3 implementation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[ANNOUNCE] Apache PredictionIO 0.11.0-incubating Release
The Apache PredictionIO team would like to announce the release of Apache PredictionIO 0.11.0-incubating. Release notes are here: https://github.com/apache/incubator-predictionio/blob/v0.11.0-incubating/RELEASE.md Apache PredictionIO (incubating) is an open source Machine Learning Server built on top of state-of-the-art open source stack, that enables developers to manage and deploy production-ready predictive services for various kinds of machine learning tasks. More details regarding Apache PredictionIO (incubating) can be found here: http://predictionio.incubator.apache.org/ The release artifacts can be downloaded here: https://dist.apache.org/repos/dist/release/incubator/predictionio/0.11.0-incubating/ All JIRAs completed for this release are tagged with 'FixVersion = 0.11.0-incubating'; the JIRA release notes can be found here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320420=12338381 Thanks! The Apache PredictionIO Team DISCLAIMER Apache PredictionIO (incubating) is an effort undergoing incubation at the Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
