Re: any plan to support "replace" function?

[hongbin ma]

Thanks Julian, it's really helpful. Just want to make sure there's no
duplicated efforts.

On Sun, Sep 18, 2016 at 12:57 AM, Julian Hyde  wrote:

> I find it frustrating when you ask “are there plans to…”. Calcite is a
> volunteer project. I don’t know what other contributors are doing, and I
> don’t control their time. If someone is working on something, they will
> have logged a JIRA case and marked it “in progress” or set the “fix
> version” field. If you had done a search, you would have found a JIRA case
> [1]; it has been inactive for 2 years.
>
> If you really want this function, contribute a patch.
>
> Julian
>
> [1] https://issues.apache.org/jira/browse/CALCITE-420 <
> https://issues.apache.org/jira/browse/CALCITE-420>
>
> > On Sep 17, 2016, at 5:49 AM, hongbin ma  wrote:
> >
> > hi experts
> >
> > recently we need to enable "replace" function for our customer, like
> > replacing all the X with Y in "123X456X". According to
> > https://calcite.apache.org/docs/reference.html the replace function
> seems
> > not implemented. I see there's another function called "OVERLAY", however
> > it's replacing with "index" instead of a "pattern".
> >
> > Is there any plans to support replace function? Do I have to implement a
> > UDF by myself now?​
> >
> > --
> > Regards,
> >
> > *Bin Mahone | 马洪宾*
>
>


-- 
Regards,

*Bin Mahone | 马洪宾*

Re: Query optimization by using Rules

[Γιώργος Θεοδωράκης]

Thank you for the quick response Julian,

I am interested mainly in logical transformation rules. I try to create an
optimized logical plan and transform it in a physical one in my engine. The
engine is streaming and uses ByteBuffers, and I am wondering if it is
possible to optimize simple queries with the following procedure:

1)create a calcite schema that my engine uses without using all the input
data. My calcite schema would have only dummy data that are a small
percentage of real data!! =>
2)validate the query and enforce logical rules to improve the logical plan
=>
3)transform this logical plan to physical in my engine

As I have seen in some of our examples, the optimizations use metadata.
However, my engine is simplified in terms of operators, as it only has one
join and some of the key operators(aggregate,filter,project,expressions). I
am not very experienced in query optimization logic and my question is if I
use dummy data, my results would be wrong because of false metadata? Should
I create an adapter like csv for the ByteBuffers that get the streaming
data (if possible?)  ?

Thanks for your time,
George

2016-09-17 19:27 GMT+03:00 Julian Hyde :

> The form of your test data depends on the kind of rules you are writing.
> If you are aiming to push down a lot of operations down to a particular
> engine (e.g. Druid can handle everything except Join) then you should run
> on that engine. If you are interested mainly in logical transformation
> rules then the capabilities of the engine are less important and you could
> run on say the Csv adapter. For convenience & familiarity I use JDBC_SCOTT,
> and if I want a larger data set with a rich model I use JDBC_FOODMART.
>
> Lastly, if you are writing unit tests for correctness of the rules, and
> don’t want to execute queries, create a sub-class of RelOptRulesTest.
>
> Julian
>
>
> > On Sep 17, 2016, at 8:08 AM, Γιώργος Θεοδωράκης 
> wrote:
> >
> > Hi,
> >
> > I am trying to create a basic planner that enforces rules on simple
> > queries. At the moment I have created a planner from the examples (and
> > samza-sql integration I found online) and used HepPlanner for testing
> some
> > rules. My question is which form should my test data be? I am using
> > something like JdbcTest.HrSchema()  right now and I am wondering if I
> > should create Tables that implement Scannable and Filterable in order to
> > implement optimizations.
> >
> > Thanks,
> > George
>
>

Re: [VOTE] Release apache-calcite-1.9.0 (release candidate 1)

[Julian Hyde]

+1

Downloaded, checked signatures, LICENSE, NOTICE, README, compiled, ran tests.

Note: When I imported your keys I got the message 

gpg: Signature made Sat Sep 17 01:05:57 2016 PDT
gpg:using RSA key 0x931E4AB3C516B444
gpg: Good signature from "Jesus Camacho Rodriguez "
gpg: WARNING: This key is not certified with a trusted signature!
gpg:  There is no indication that the signature belongs to the owner.

This means that you are not part of the “web of trust”. You should join it when 
you get the chance, best done face-to-face. Note a show-stopper, but best done 
before the release is announced. See [1].

Julian

[1] https://www.apache.org/dev/release-signing#link-into-wot 


> On Sep 17, 2016, at 4:49 AM, Jesus Camachorodriguez  
> wrote:
> 
> Hi all,
> 
> I have created a build for Apache Calcite 1.9.0, release candidate 1.
> 
> Thanks to everyone who has contributed to this release.
> You can read the release notes here:
> https://github.com/apache/calcite/blob/calcite-1.9.0/site/_docs/history.md
> 
> The commit to be voted upon:
> http://git-wip-us.apache.org/repos/asf/calcite/commit/08c56b1
> 
> Its hash is 08c56b158ffcfcf205a919cc9fff77a692e649f6.
> 
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.9.0-rc1/
> 
> The hashes of the artifacts are as follows:
> src.tar.gz.md5 b3b6a15671b1e8ffd7f989398e050de4
> src.tar.gz.sha1 8f17b2d7896bac6123ae4a1491e04f4960094a59
> src.zip.md5 7c66353701d1924a9e2656bd76e570cb
> src.zip.sha1 8a5c7d27b0240cbcacd565eadf451de9dff6d2fa
> 
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachecalcite-1024
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jcamacho.asc
> 
> Please vote on releasing this package as Apache Calcite 1.9.0.
> 
> The vote is open for the next 96 hours and passes if a majority of
> at least three +1 PMC votes are cast.
> 
> [ ] +1 Release this package as Apache Calcite 1.9.0
> [ ]  0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
> 
> 
> Here is my vote:
> 
> +1 (binding)
> 
> Jesús
> 

Re: any plan to support "replace" function?

[Julian Hyde]

I find it frustrating when you ask “are there plans to…”. Calcite is a 
volunteer project. I don’t know what other contributors are doing, and I don’t 
control their time. If someone is working on something, they will have logged a 
JIRA case and marked it “in progress” or set the “fix version” field. If you 
had done a search, you would have found a JIRA case [1]; it has been inactive 
for 2 years.

If you really want this function, contribute a patch.

Julian

[1] https://issues.apache.org/jira/browse/CALCITE-420 


> On Sep 17, 2016, at 5:49 AM, hongbin ma  wrote:
> 
> hi experts
> 
> recently we need to enable "replace" function for our customer, like
> replacing all the X with Y in "123X456X". According to
> https://calcite.apache.org/docs/reference.html the replace function seems
> not implemented. I see there's another function called "OVERLAY", however
> it's replacing with "index" instead of a "pattern".
> 
> Is there any plans to support replace function? Do I have to implement a
> UDF by myself now?​
> 
> -- 
> Regards,
> 
> *Bin Mahone | 马洪宾*

Apache Calcite as a "job creator"

[Vladimir Sitnikov]

Hi,

I just did a random "Calcite" search on upwork.com and it turned out
there's a research job to apply Calcite to a real project.

https://www.upwork.com/jobs/_~01e957c78cc8972025/

>Work on a 5-7 week PoC to use Apache Calcite (or other tools proposed)
>to map between a Legacy and Target SQL Server data model.
>The "Database Intermediary" would receive statements (CRUD)
>from the client Java application and transform the statement
>on the fly to the statement(s) required to update the SQL Server
>database according to the Target data model

Great work, everybody!

PS. Feel free to apply for the project. I would consider participating
provided I had more spare time.

Vladimir

Re: Query optimization by using Rules

[Julian Hyde]

The form of your test data depends on the kind of rules you are writing. If you 
are aiming to push down a lot of operations down to a particular engine (e.g. 
Druid can handle everything except Join) then you should run on that engine. If 
you are interested mainly in logical transformation rules then the capabilities 
of the engine are less important and you could run on say the Csv adapter. For 
convenience & familiarity I use JDBC_SCOTT, and if I want a larger data set 
with a rich model I use JDBC_FOODMART.

Lastly, if you are writing unit tests for correctness of the rules, and don’t 
want to execute queries, create a sub-class of RelOptRulesTest.

Julian


> On Sep 17, 2016, at 8:08 AM, Γιώργος Θεοδωράκης  
> wrote:
> 
> Hi,
> 
> I am trying to create a basic planner that enforces rules on simple
> queries. At the moment I have created a planner from the examples (and
> samza-sql integration I found online) and used HepPlanner for testing some
> rules. My question is which form should my test data be? I am using
> something like JdbcTest.HrSchema()  right now and I am wondering if I
> should create Tables that implement Scannable and Filterable in order to
> implement optimizations.
> 
> Thanks,
> George

Query optimization by using Rules

[Γιώργος Θεοδωράκης]

Hi,

I am trying to create a basic planner that enforces rules on simple
queries. At the moment I have created a planner from the examples (and
samza-sql integration I found online) and used HepPlanner for testing some
rules. My question is which form should my test data be? I am using
something like JdbcTest.HrSchema()  right now and I am wondering if I
should create Tables that implement Scannable and Filterable in order to
implement optimizations.

Thanks,
George

Re: [VOTE] Release apache-calcite-1.9.0 (release candidate 0)

[Jesus Camacho Rodriguez]

I think I made some mistake cleaning my local repo, sorry about that...

Cancelling the vote, I will create a new RC and start a new vote shortly.

Thanks,
Jesús




On 9/17/16, 1:43 AM, "Julian Hyde"  wrote:

>-1 (binding)
>
>Downloaded, checked hashes, LICENSE, NOTICE; built on OS X using JDK 1.8. OK 
>so far.
>
>I ran the tests and had one test failure:
>
>Tests run: 267, Failures: 1, Errors: 0, Skipped: 20, Time elapsed: 57.83 sec 
><<< FAILURE! - in org.apache.calcite.test.JdbcTest
>testVersion(org.apache.calcite.test.JdbcTest)  Time elapsed: 0.003 sec  <<< 
>FAILURE!
>java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at org.apache.calcite.test.JdbcTest.testVersion(JdbcTest.java:1054)
>
>Other problems:
>* README says “1.8.0”, should say “1.9.0”
>* Ditto howto.md
>* In pom.xml, version.minor is 8.
>
>The last of these cause the JDBC driver to return the wrong version, and 
>explains the failure of JdbcTest.testVersion, and is a show-stopper.
>
>Julian
>
>
>> On Sep 16, 2016, at 2:21 PM, Jesus Camachorodriguez  
>> wrote:
>> 
>> Hi all,
>> 
>> I have created a build for Apache Calcite 1.9.0, release candidate 0.
>> 
>> Thanks to everyone who has contributed to this release.
>> You can read the release notes here:
>> https://github.com/apache/calcite/blob/calcite-1.9.0/site/_docs/history.md
>> 
>> The commit to be voted upon:
>> http://git-wip-us.apache.org/repos/asf/calcite/commit/e371a39
>> 
>> Its hash is e371a3950094cbf3d974b12f7df87dff4caf47a7.
>> 
>> The artifacts to be voted on are located here:
>> https://dist.apache.org/repos/dist/dev/calcite/apache-calcite-1.9.0-rc0/
>> 
>> The hashes of the artifacts are as follows:
>> src.tar.gz.md5 7f2b61663072de86134ce6fa07ce322a
>> src.tar.gz.sha1 0c5fe8f91cc1350142a08673e501e45970162411
>> src.zip.md5 b2ab48c904bf2f2df2f3485e72ef6e32
>> src.zip.sha1 72af6dfd8f191a3e513f87b7511808744cd4998b
>> 
>> A staged Maven repository is available for review at:
>> https://repository.apache.org/content/repositories/orgapachecalcite-1023
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/jcamacho.asc
>> 
>> Please vote on releasing this package as Apache Calcite 1.9.0.
>> 
>> The vote is open for the next 96 hours and passes if a majority of
>> at least three +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Calcite 1.9.0
>> [ ]  0 I don't feel strongly about it, but I'm okay with the release
>> [ ] -1 Do not release this package because...
>> 
>> 
>> Here is my vote:
>> 
>> +1 (binding)
>> 
>> Jesús
>> 
>> 
>> 
>
>