Whats JVM size given for Kylin? Try with 16 GB Memory mentioned in setenv.sh file. You need to use max. JVM for loading billions of data.
Regards, Manoj From: Ruslan Dautkhanov [mailto:[email protected]] Sent: Tuesday, January 09, 2018 12:59 PM To: [email protected] Subject: Re: cachesync.Broadcaster:172 : error running wiping java.lang.IllegalArgumentException Hi Billy, Thank you - that was it. Although it was just hiding real problem - with jvm memory [1]. I have requested to build a cube on 12b records and can't now make Kylin back to stable state. Kylin crashes on startup. I have already tried to tune up Xmx and such but still no luck [2]. [1] INFO: Starting ProtocolHandler ["ajp-bio-9009"] Jan 09, 2018 12:09:49 AM org.apache.catalina.startup.Catalina start INFO: Server startup in 12762 ms java.lang.OutOfMemoryError: Requested array size exceeds VM limit Dumping heap to java_pid60827.hprof ... Heap dump file created [1731330948 bytes in 2.963 secs] # # java.lang.OutOfMemoryError: Requested array size exceeds VM limit # -XX:OnOutOfMemoryError="kill -9 %p" # Executing /bin/sh -c "kill -9 60827"... Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release Error occurred during initialization of VM java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at java.lang.ref.Reference.<clinit>(Reference.java:232) [2] export KYLIN_JVM_SETTINGS="-Xms2g -Xmx16g -Xss2g -XX:MaxPermSize=256M -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M" -- Ruslan Dautkhanov On Mon, Jan 8, 2018 at 8:29 PM, Billy Liu <[email protected]<mailto:[email protected]>> wrote: Hello Ruslan, Could you check the "kylin.server.cluster-servers". There should no "http://"; as prefix. 2018-01-09 0:59 GMT+08:00 Ruslan Dautkhanov <[email protected]<mailto:[email protected]>>: Below error crashes Kylin. Attempts to start Kylin back results in the same exception. This started happening after a new cube build was scheduled.. Is this a known bug and any ideas how to work around this ? 2018-01-08 09:54:06,427 INFO [Scheduler 1698178565 Job b595302c-35d4-45fd-a895-6766747eeae7-81] cube.CubeManager:358 : Updating cube instance 'mv2cube' 2018-01-08 09:54:06,430 WARN [Scheduler 1698178565 Job b595302c-35d4-45fd-a895-6766747eeae7-81] model.Segments:421 : NEW segment start does not fit/connect with other segments: mv2cube[20100101000000_20180101000000] 2018-01-08 09:54:06,430 WARN [Scheduler 1698178565 Job b595302c-35d4-45fd-a895-6766747eeae7-81] model.Segments:423 : NEW segment end does not fit/connect with other segments: mv2cube[20100101000000_20180101000000] 2018-01-08 09:54:06,454 INFO [Scheduler 1698178565 Job b595302c-35d4-45fd-a895-6766747eeae7-81] cli.DictionaryGeneratorCLI:57 : Building dictionary for MV_PROD.CONVERTED_FACT.INDIVID 2018-01-08 09:54:06,454 DEBUG [pool-7-thread-1] cachesync.Broadcaster:141 : Servers in the cluster: [http://pc1udatahad03:7070] 2018-01-08 09:54:06,466 ERROR [pool-7-thread-1] cachesync.Broadcaster:172 : error running wiping java.lang.IllegalArgumentException: URI: http://pc1udatahad03:7070 -- does not match pattern (?:([^:]+)[:]([^@]+)[@])?([^:]+)(?:[:](\d+))? at org.apache.kylin.common.restclient.RestClient.<init>(RestClient.java:91) at org.apache.kylin.metadata.cachesync.Broadcaster$1.run(Broadcaster.java:144) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
