+1 .
Good job!
At 2020-10-14 15:12:15, "Xiaoxiang Yu" wrote:
>Hi all,
>
>
>
>I have created a build for Apache Kylin 3.1.1, release candidate 1.
>
>
>
>
>
>Changes highlights:
>
>
>
>[KYLIN-4612] - Support job status write to kafka
>
>
>
>[KYLIN-4712] - Optimize
Contratulations !
At 2020-10-14 22:25:10, "ShaoFeng Shi" wrote:
>The Project Management Committee (PMC) for Apache Kylin has invited Rupeng
>Wang (王汝鹏, wangrup...@apache.org) to become a committer and we are pleased
>to announce that he has accepted.
>
>Being a committer enables
1.读写锁改成分桶和自旋锁，允许瞬时的脏读。
2.更新元数据时不再广播自己，即修改元数据的进程。需要靠广播刷新的缓存，在更新操作时同步刷新。
3.缓存更新细粒度，按最小原子进行更新，考虑记录变更的版本号时间戳
依次往下修改，直到满足设计目标。
在 2020-07-22 05:27:24，"chuxiao" 写道：
>kylin当前的缓存机制，比较适合cube数几十上百，不频繁更新的场景。
>当有2千以上的cube，每天更新几百个，metadata上的读写锁，任何修改全更新的广播机制，导致一旦有持续的元数据更新，整个集群响应
kylin当前的缓存机制，比较适合cube数几十上百，不频繁更新的场景。
当有2千以上的cube，每天更新几百个，metadata上的读写锁，任何修改全更新的广播机制，导致一旦有持续的元数据更新，整个集群响应时间大幅拉长，建模和查询频繁出现超时。
KYLIN-4169缓解了这个问题，但还不够。
我想重新设计缓存功能，设计目标是在单个项目1万个cube，每天新建/删除一千个的场景下，建模和查询不会出现频繁超时，保持系统可用性。
create jira KYLIN-4654
在 2020-07-22 05:39:49，"chuxiao" 写道：
>1.读写锁改成分桶和自旋锁，允许瞬时的脏读。
>2.更新元数据时不再广播自己，即修改元数据的进程。需要靠广播刷新的缓存，在更新操作时同步刷新。
>3.缓存更新细粒度，按最小原子进行更新，考虑记录变更的版本号时间戳
>依次往下修改，直到满足设计目标。
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
is works as you
>> expect, if your idea is proved works in your production env, maintainer will
>> be happy to learn and view your code change.
>>Thank you very much and good luck to you.
>>
>>
>>
>>
>>
>>
>>
>>
>
${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true
thanks.
At 2020-08-04 11:03:02, "shylinzhang" wrote:
Dear all,
I found many tmp files in HDFS tmp dictionary, and I checked the kylin conf
file and didn’t find any tmp route in them.
I found
mark sure you can connection the "EMR cluster hostname":7070，
maybe network can not connection.
At 2020-07-13 11:58:34, "xatax" wrote:
>I have attempted to connect to Kylin using jdbc driver for the following
>Kylin versions 2.6, 3.0.2 and 3.1 and have been unable to make
In kylin-server-log4j.properties and kylin-tools-log4j.properties,
log4j.logger.org.apache.kylin=DEBUG,file
Only version is less than 1.0, or new feature package use debug level for
default.
Could we change default log level to INFO?
Will 3.x continue to release？For example，support hbase rsgroup.
At 2020-07-24 19:23:11, "ShaoFeng Shi" wrote:
Hello, Kylin users,
Regarding the Kylin Parquet storage, we hope to update the progress here. At
present, we have completed the main development work[1], design
+1 from my side.
At 2021-02-03 21:31:34, "Xiaoxiang Yu" wrote:
>Hi all,
>
>
>
>
>I have created a build for Apache Kylin 4.0.0-beta, release candidate 2.
>Please note that this release is built on kylin-on-parquet-v2 branch.
>
>
>
>
>Changes highlights:
>
>
>
>
>[KYLIN-4857] -
+1 (binding)
At 2024-01-21 00:12:42, "Li Yang" wrote:
>Hi all,
>
>I have created a build for Apache Kylin 4.0.4, release candidate 1. This is
>a very small release aiming to upgrade the versions of dependent components
>mainly.
>
>Changes highlights:
>
> - Bump commons-fileupload
chuxiao created KYLIN-4163:
--
Summary: CreateFlatHiveTableStep has not yarn app url when hive
job running
Key: KYLIN-4163
URL: https://issues.apache.org/jira/browse/KYLIN-4163
Project: Kylin
Issue
chuxiao created KYLIN-4250:
--
Summary: FechRunnner should skip the job to process other jobs
instead of throwing exception when the job section metadata is not found
Key: KYLIN-4250
URL: https://issues.apache.org/jira
chuxiao created KYLIN-4205:
--
Summary: configuration support set hive.intermediate-table-prefix
Key: KYLIN-4205
URL: https://issues.apache.org/jira/browse/KYLIN-4205
Project: Kylin
Issue Type
chuxiao created KYLIN-4211:
--
Summary: PartitionDesc support custom year、month、day partitions
name
Key: KYLIN-4211
URL: https://issues.apache.org/jira/browse/KYLIN-4211
Project: Kylin
Issue Type
chuxiao created KYLIN-4210:
--
Summary: Support the custom year, month, and day field name
partition for the fact table
Key: KYLIN-4210
URL: https://issues.apache.org/jira/browse/KYLIN-4210
Project: Kylin
chuxiao created KYLIN-4190:
--
Summary: hiveproducer send data excetpion because hive mertics
table location path prefix is different with defaut fs when hdfs uses
router-based federation
Key: KYLIN-4190
URL: https
chuxiao created KYLIN-4728:
--
Summary: hive global dict optimize
Key: KYLIN-4728
URL: https://issues.apache.org/jira/browse/KYLIN-4728
Project: Kylin
Issue Type: Improvement
Reporter
chuxiao created KYLIN-4654:
--
Summary: new metadata read/write and reload mechanism
Key: KYLIN-4654
URL: https://issues.apache.org/jira/browse/KYLIN-4654
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4678:
--
Summary: continue execute job when StorageCleanupJob sub step has
error
Key: KYLIN-4678
URL: https://issues.apache.org/jira/browse/KYLIN-4678
Project: Kylin
Issue
chuxiao created KYLIN-4677:
--
Summary: StorageCleanupJob throw NPE
Key: KYLIN-4677
URL: https://issues.apache.org/jira/browse/KYLIN-4677
Project: Kylin
Issue Type: Bug
Reporter: chuxiao
chuxiao created KYLIN-4679:
--
Summary: clean hive table support hive table prefix
Key: KYLIN-4679
URL: https://issues.apache.org/jira/browse/KYLIN-4679
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4671:
--
Summary: modify FetchRunner catch getOutput NPE log level to debug
for ignore too many logs
Key: KYLIN-4671
URL: https://issues.apache.org/jira/browse/KYLIN-4671
Project: Kylin
chuxiao created KYLIN-4686:
--
Summary: clean metadata support to delete all jobs
Key: KYLIN-4686
URL: https://issues.apache.org/jira/browse/KYLIN-4686
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4685:
--
Summary: return user friendly msg when stackoverflowerror
Key: KYLIN-4685
URL: https://issues.apache.org/jira/browse/KYLIN-4685
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4687:
--
Summary: add unify clean sh to excute some clean shells
Key: KYLIN-4687
URL: https://issues.apache.org/jira/browse/KYLIN-4687
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4693:
--
Summary: avoid NPE when HDFSPathGarbageCollectionStep run
Key: KYLIN-4693
URL: https://issues.apache.org/jira/browse/KYLIN-4693
Project: Kylin
Issue Type: Bug
chuxiao created KYLIN-4610:
--
Summary: updatet kylin.engine.livy.backtick.quote default
Key: KYLIN-4610
URL: https://issues.apache.org/jira/browse/KYLIN-4610
Project: Kylin
Issue Type: Bug
chuxiao created KYLIN-4609:
--
Summary: setenv.sh add zgc config for big memory
Key: KYLIN-4609
URL: https://issues.apache.org/jira/browse/KYLIN-4609
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4608:
--
Summary: add deletecubefast api for delete 300 cubes fast
Key: KYLIN-4608
URL: https://issues.apache.org/jira/browse/KYLIN-4608
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4613:
--
Summary: add buildCubeCLi as hadoop main class and jobRestClient
Key: KYLIN-4613
URL: https://issues.apache.org/jira/browse/KYLIN-4613
Project: Kylin
Issue Type
chuxiao created KYLIN-4614:
--
Summary: Broadcaster node exclude itself
Key: KYLIN-4614
URL: https://issues.apache.org/jira/browse/KYLIN-4614
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4612:
--
Summary: job status write to kafka
Key: KYLIN-4612
URL: https://issues.apache.org/jira/browse/KYLIN-4612
Project: Kylin
Issue Type: Improvement
Components
chuxiao created KYLIN-4611:
--
Summary: modify PATTERN_SPARK_APP_URL to Tracking URL
Key: KYLIN-4611
URL: https://issues.apache.org/jira/browse/KYLIN-4611
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4635:
--
Summary: set org.apache.kylin default log level info
Key: KYLIN-4635
URL: https://issues.apache.org/jira/browse/KYLIN-4635
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4642:
--
Summary: load hive env classpath when only kylin first start
Key: KYLIN-4642
URL: https://issues.apache.org/jira/browse/KYLIN-4642
Project: Kylin
Issue Type
chuxiao created KYLIN-4626:
--
Summary: add set kylin home sh
Key: KYLIN-4626
URL: https://issues.apache.org/jira/browse/KYLIN-4626
Project: Kylin
Issue Type: Improvement
Reporter
chuxiao created KYLIN-4606:
--
Summary: add logs when query olap and pushdown botherror
Key: KYLIN-4606
URL: https://issues.apache.org/jira/browse/KYLIN-4606
Project: Kylin
Issue Type: Improvement
chuxiao created KYLIN-4607:
--
Summary: destributed scheduler avoid change job status when
fetcherRunner error
Key: KYLIN-4607
URL: https://issues.apache.org/jira/browse/KYLIN-4607
Project: Kylin
chuxiao created KYLIN-4605:
--
Summary: HiveProducer write metrics less when one host running
some kylin servers
Key: KYLIN-4605
URL: https://issues.apache.org/jira/browse/KYLIN-4605
Project: Kylin
chuxiao created KYLIN-4603:
--
Summary: listjob return NPE
Key: KYLIN-4603
URL: https://issues.apache.org/jira/browse/KYLIN-4603
Project: Kylin
Issue Type: Bug
Components: Job Engine
chuxiao created KYLIN-4837:
--
Summary: optimize CubeMigrationCLI
Key: KYLIN-4837
URL: https://issues.apache.org/jira/browse/KYLIN-4837
Project: Kylin
Issue Type: Improvement
Reporter
chuxiao created KYLIN-4838:
--
Summary: fix KYLIN-4679 bug
Key: KYLIN-4838
URL: https://issues.apache.org/jira/browse/KYLIN-4838
Project: Kylin
Issue Type: Improvement
Reporter: chuxiao
chuxiao created KYLIN-4836:
--
Summary: fix CubeMigrationCLI bug
Key: KYLIN-4836
URL: https://issues.apache.org/jira/browse/KYLIN-4836
Project: Kylin
Issue Type: Improvement
Reporter
45 matches
Mail list logo
