[jira] [Created] (KYLIN-5704) For ‘in’ condition query of non-time partition columns, when the data type of the value in 'in' condition is inconsistent with that of the non-time partition column, the

2023-11-14 Thread Hrongrong Cao (Jira)
Hrongrong Cao created KYLIN-5704:


 Summary: For ‘in’ condition query of non-time partition columns, 
when the data type of the value in 'in' condition is inconsistent with that of 
the non-time partition column, the segment pruner fails, resulting in full 
Segment scanning
 Key: KYLIN-5704
 URL: https://issues.apache.org/jira/browse/KYLIN-5704
 Project: Kylin
  Issue Type: Bug
Affects Versions: 5.0-alpha
Reporter: Hrongrong Cao
 Fix For: 5.0-beta


The query column is a non-time partition column, a common dimension column, and 
the filter condition of the common dimension column is col in (x1, x2...) In 
this case (and because the col and x1 types do not match, it is automatically 
converted to (cast col as string) in (x1,x2..), Fileprunner will report an 
error because 
org.apache.spark.sql.execution.datasource.FilePruner#convertCastFilter does not 
handle in.

Explain that the convertCastFilter method is to remove the cast condition, so 
that the filter condition can be matched when calling 
DataSourceStrategy.translateFilter, and then the Segment can be filtered. 
However, currently convertCastFilter misses the processing of the in condition, 
so translateFilter cannot match and becomes empty, so The query was thrown 
incorrectly.

In addition: if it is a time partition column, it does not matter if an error 
is reported here, because in the previous steps, the calcite file prunner has 
already completed the Segment Prune of the time partition column.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Default Hive user/password of docker Kylin 5.0

2023-11-14 Thread Nam Đỗ Duy
Thank you Xiaoxiang,

Do I need to add port 1 of Hive to the docker run when creating
Container like:

docker run -d -m 8G -p 1:1 7070:7070 -p 8088:8088 -p 50070:50070 -p
8032:8032 -p 8042:8042 -p 2181:2181
apachekylin/apache-kylin-standalone:5.0-beta

On Tue, Nov 14, 2023 at 10:00 AM Xiaoxiang Yu  wrote:

>
> https://github.com/apache/kylin/blob/8de5c7a7121dc37729a12ee231041f8d89d1494c/dev-support/release-manager/standalone-docker/all_in_one/entrypoint.sh#L83C48-L83C48
>
> 
> With warm regard
> Xiaoxiang Yu
>
>
>
> On Mon, Nov 13, 2023 at 7:18 PM Nam Đỗ Duy  wrote:
>
> > Dear Team,
> >
> > I am using scala to connect to Hive after install Kylin docker 5.0,
> please
> > kindly tell me the default HIVE user/password to fill to this code:
> >
> > Thank you very much and best regards
> >
> > // JDBC URL to connect to Hive
> > val jdbcURL = "jdbc:hive2://localhost:1/your_database"
> >
> > // Hive connection properties
> > val connectionProperties = new java.util.Properties()
> > connectionProperties.setProperty("user", "your_username")
> > connectionProperties.setProperty("password", "your_password")
> >
> > // JDBC driver name and database URL
> > val driverName = "org.apache.hive.jdbc.HiveDriver"
> >
>


Re: Why we choose Kylin

2023-11-14 Thread Nam Đỗ Duy
Thank you very much for your information, will catch up soon

On Tue, Nov 14, 2023 at 1:40 PM Xiaoxiang Yu  wrote:

> I have to say currently Kylin is focused on OLAP solutions, without very
> few use cases in data science or AI.
> And it requires a good understanding of Hadoop/Spark if you want to
> optimize query performance.
>
> For your question, I found a video of a Kylin Meetup in Shanghai(using
> English). The speaker is from eBay,
> the creator of the Kylin project, I think she has a better understanding
> than me.
>
> The second section of the video summarized the use case of Kylin in eBay,
> starting at 09:48 .
> This video also introduces the history of the Kylin project.
>
> Here is the link:
>
>
> https://www.bilibili.com/video/BV17h41127bV/?spm_id_from=333.337.search-card.all.click_source=233a70cff82cc278ec07b1660fdbc7d2
>
> 
> With warm regard
> Xiaoxiang Yu
>
>
>
> On Tue, Nov 14, 2023 at 1:42 PM Nam Đỗ Duy  wrote:
>
> > Thank you Xiaoxiang for your reply
> >
> > This may be final question for our Board of Management to decide to use
> > Kylin so please kindly answer:
> >
> > During your work with those big firms (listed in Who is using Kylin),
> what
> > do you find those big firms are utilizing kylin in their work in terms of
> > (for example):
> >
> > - business area (like Marketing, research, sales support, risk management
> > etc)
> > - functional teams (data analyst, data science, AI project)
> > - scope and scale of the project
> > -…
> >
> > Thank you very much
> >
> > On Tue, 14 Nov 2023 at 09:58 Xiaoxiang Yu  wrote:
> >
> > > Yes, we have meetups in different cities, it can be searched. Besides,
> we
> > > communicate by email or zoom.
> > > 
> > > With warm regard
> > > Xiaoxiang Yu
> > >
> > >
> > >
> > > On Tue, Nov 14, 2023 at 12:27 AM Nam Đỗ Duy 
> > > wrote:
> > >
> > > > Hi Xiaoxiang,
> > > >
> > > > Please kindly let us know:
> > > >
> > > > My boss is asking that: how do you know the companies in “Who is
> using
> > > > Kylin” section are actually using Kylin…do they inform you that fact
> or
> > > do
> > > > you have any agreement with them?
> > > >
> > > > Thank you very much
> > > >
> > > > On Mon, 13 Nov 2023 at 09:34 Xiaoxiang Yu  wrote:
> > > >
> > > > > 1. Which companies are using kylin now
> > > > > You can visit the home page https://kylin.apache.org, and go to
> 'Who
> > > is
> > > > > using Kylin?' part,
> > > > > you will be finding logos of these companies.
> > > > >
> > > > > 2. How do they use kylin’s capabilities in AI/ML projects?
> > > > > Currently I am focusing on Kylin itself. I did not have enough
> > > knowledge
> > > > in
> > > > > AI/ML.
> > > > > Here is what I know.
> > > > > As far as I know, Kylin used to provide a Python library, so Kylin
> > can
> > > be
> > > > > integrated
> > > > > with some Python ML tools(such as
> > > > > https://docs.byzer.org/#/byzer-lang/en-us/
> > > > > ),
> > > > > but I don't know if it still works at the moment. I think it needs
> > some
> > > > > test and modification
> > > > > to make these work with Kylin 5.
> > > > >
> > > > >
> > > > >
> > > > > 
> > > > > With warm regard
> > > > > Xiaoxiang Yu
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Nov 13, 2023 at 10:00 AM Nam Đỗ Duy  >
> > > > > wrote:
> > > > >
> > > > > > Hi Xiaoxiang
> > > > > >
> > > > > > Regarding the reason why we should choose kylin please provide
> real
> > > > > > use-cases to help me answer our boss’s question:
> > > > > >
> > > > > > 1. Which companies are using kylin now
> > > > > > 2. How do they use kylin’s capabilities in AI/ML projects
> > > > > >
> > > > > > Thank you very much
> > > > > >
> > > > > > On Mon, 6 Nov 2023 at 13:42 Xiaoxiang Yu 
> wrote:
> > > > > >
> > > > > > > Here are some blogs which can help you to introduce advantages
> of
> > > > > Kylin .
> > > > > > >
> > > > > > > -
> https://kylin.apache.org/blog/2022/01/12/The-Future-Of-Kylin/
> > > > > > > -
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KYLIN/Why+did+Youzan+choose+Kylin+4
> > > > > > > - https://kylin.apache.org/blog/
> > > > > > >
> > > > > > >
> > > > > > > 
> > > > > > > With warm regard
> > > > > > > Xiaoxiang Yu
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Nov 3, 2023 at 4:12 PM Nam Đỗ Duy
>  > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Dear Sir/Madam
> > > > > > > >
> > > > > > > > I am persuading my company to use Kylin as OLAP platform.
> > > > > > > >
> > > > > > > > Could you please give some fact or some document to my
> > > presentation
> > > > > > about
> > > > > > > > the reason why we should choose Kylin comparing with other
> OLAP
> > > > > > platform.
> > > > > > > >
> > > > > > > > Thank you very much
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>