I think it's a bug. Thank you for digging into the code. Would you like to submit a PR to fix it?
With Warm regards Billy Liu 2018-03-05 3:32 GMT-08:00 余楚倩 <[email protected]>: > > > Hi, kylin developers. My cube building procedure failing at Step 2 > "Redistribute Flat Hive Table" because Kylin always trying to submit a mr job > to the default yarn queue. > > I have overrided the mapred.job.queue.name property in both > kylin_hive_conf.xml and kylin.properties but it doesn't work. > > kylin.properties > ``` > kylin.source.hive.beeline-params=-n hive -p hive --hiveconf > mapred.job.queue.name=myQueue -u > "jdbc:hive2://myZk:2181/;serviceDiscoveryMode=zooKeeper;" > ``` > > kylin_hive_conf.xml > ``` > <property> > <name>mapred.job.queue.name</name> > <value>myQueue</value> > </property> > ``` > > After digging into the source code ,I found that kylin try to get the row > count of hive table before redistributing it. But it dose not override the > hive configuration when using jdbc to connect to hive server. I wonder if it > is a bug or there is some solutions? > > ``` > public BeelineHiveClient(String beelineParams) { > if (StringUtils.isEmpty(beelineParams)) { > throw new IllegalArgumentException("BeelineParames cannot be > empty"); > } > String[] splits = StringUtils.split(beelineParams); > String url = null, username = null, password = null; > for (int i = 0; i < splits.length; i++) { > if ("-u".equals(splits[i])) { > url = stripQuotes(splits[i + 1]); > } > if ("-n".equals(splits[i])) { > username = stripQuotes(splits[i + 1]); > } > if ("-p".equals(splits[i])) { > password = stripQuotes(splits[i + 1]); > } > } > this.init(url, username, password); > } > > private void init(String url, String username, String password) { > try { > Class.forName("org.apache.hive.jdbc.HiveDriver"); > cnct = DriverManager.getConnection(url, username, password); > //use DriverManager.getConnection(url,properites) here to override hive > configuration settings? > stmt = cnct.createStatement(); > metaData = cnct.getMetaData(); > } catch (SQLException | ClassNotFoundException e) { > throw new RuntimeException(e); > } > } > ```
