메뉴 건너뛰기

검색창 닫기

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.


spark spark2.0.0에서 hive 2.0.1 table을 읽어 출력하는 예제 소스(HiveContext, SparkSession, SQLContext)

총관리자 2017.03.09 14:57 조회 수 : 163

* 주의 : sQLContext, HiveContext는 "The type HiveContext is deprecated" 상태임

* 주의 : HiveContext는 window함수를 사용할수 있지만(예, SELECT value, dense_rank() OVER (PARTITION BY key ORDER BY value DESC) as rank FROM src) SQLContext는 사용할 수 없음


1. hive table 생성및 데이타 로드, 확인

hive> create table test_table(name varchar(100), age int)

row format delimited fields terminated by ','

lines terminated by 'n';


hive> load data local inpath '/tmp/test.txt' into table test_table;

hive> select * from test_table;
OK
park 10
sang 20
hyeon 30


2. spark에서 hive테이블을 다루기 위한 dependency설정

 가. pom.xml파일에 dependency추가

   <dependency>      <groupId>org.apache.spark</groupId>      <artifactId>spark-hive_2.11</artifactId>      <version>2.0.0</version>      <scope>provided</scope>    </dependency>


3. 여러가지 경우의 샘플소스

  가. 테스트 메서드를 호출하는 부분

  public static void main() {

    SparkConf sparkConf=new SparkConf().setAppName("JavaSparkTest");

    JavaSparkContext jsc=new JavaSparkContext(sparkConf);

    test14(jsc);

  }


1) HiveContext를 이용하는 경우
  static void test14(JavaSparkContext sc) {
   HiveContext ctx = new HiveContext(sc);
   Dataset<Row>  rows = ctx.sql("select * from default.test_table");
 
   Dataset<String> stringsDS = rows.map(new MapFunction<Row, String>() {
    @Override
    public String call(Row row) throws Exception {
     //return "Key : "+row.get(0) + ", Value : "+row.get(1);
     return "Value : "+row.get(0);
    }
   }, Encoders.STRING());
  
   System.out.println("result #1 =>"+stringsDS.toJavaRDD().collect());
  }
---> 실행결과
result #1 =>[Value : park, Value : sang, Value : hyeon] 
 
2) SparkSession을 이용하는 경우 
  static void test15() {
   SparkSession session = new SparkSession.Builder().appName("SparkJoinExample").master("local").enableHiveSupport().getOrCreate();
      Dataset<Row> dset = session.sql("select * from default.test_table");
     
   System.out.println("result #2 =>"+dset.toJavaRDD().collect());
  }
--->실행결과
result #2 =>[[park], [sang], [hyeon]] 


3) SQLContext를 이용하는 경우
  static void test16(JavaSparkContext sc) {
   SQLContext ctx = new SQLContext(sc);
   Dataset<Row>  rows = ctx.sql("select * from default.test_table");
 
   Dataset<String> stringsDS = rows.map(new MapFunction<Row, String>() {
    @Override
    public String call(Row row) throws Exception {
     //return "Key : "+row.get(0) + ", Value : "+row.get(1);
     return "Value : "+row.get(0);
    }
   }, Encoders.STRING());
  
   System.out.println("result #3 =>"+stringsDS.toJavaRDD().collect());
  }
--->실행결과
result #3 =>[Value : park, Value : sang, Value : hyeon] 



--------------------HiveContext를 사용할때 아래와 같은 경고및 에러 메세지가 발생하지만.. 결과는 나온다...-----

[2017-03-09 18:35:28,674] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:29,205] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:29,205] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:30,239] [metastore.RetryingHMSHandler] [invoke(#159)] [ERROR] AlreadyExistsException(message:Database default already exists)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:891)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy28.create_database(Unknown Source)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:644)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
        at com.sun.proxy.$Proxy29.createDatabase(Unknown Source)
        at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:306)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply$mcV$sp(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:262)
        at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:209)
        at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:208)
        at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:251)
        at org.apache.spark.sql.hive.client.HiveClientImpl.createDatabase(HiveClientImpl.scala:290)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply$mcV$sp(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:72)
        at org.apache.spark.sql.hive.HiveExternalCatalog.createDatabase(HiveExternalCatalog.scala:98)
        at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase(SessionCatalog.scala:147)
        at org.apache.spark.sql.catalyst.catalog.SessionCatalog.<init>(SessionCatalog.scala:89)
        at org.apache.spark.sql.hive.HiveSessionCatalog.<init>(HiveSessionCatalog.scala:51)
        at org.apache.spark.sql.hive.HiveSessionState.catalog$lzycompute(HiveSessionState.scala:49)
        at org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:48)
        at org.apache.spark.sql.hive.HiveSessionState$$anon$1.<init>(HiveSessionState.scala:63)
        at org.apache.spark.sql.hive.HiveSessionState.analyzer$lzycompute(HiveSessionState.scala:63)
        at org.apache.spark.sql.hive.HiveSessionState.analyzer(HiveSessionState.scala:62)
        at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:49)
        at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
        at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:682)
        at com.gooper.icbms.sda.kafka.onem2m.JavaSparkPi.test14(JavaSparkPi.java:523)
        at com.gooper.icbms.sda.kafka.onem2m.JavaSparkPi.main(JavaSparkPi.java:165)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)


이 게시물을
목록
번호 제목 글쓴이 날짜 조회 수
441 Caused by: java.sql.SQLNonTransientConnectionException: Could not read resultset: unexpected end of stream, read 0 bytes from 4 오류시 확인/조치할 내용 총관리자 2016.10.31 3787
440 mybatis와 spring을 org.apache.commons.dbcp2.BasicDataSource의 DataSource로 연동할때 DB설정(참고) 총관리자 2016.10.31 992
439 How-to: Tune Your Apache Spark Jobs (Part 2) file 총관리자 2016.10.31 77
438 How-to: Build a Complex Event Processing App on Apache Spark and Drools file 총관리자 2016.10.31 253
437 Flume을 이용한 데이타 수집시 HBase write 성능 튜닝 file 총관리자 2016.10.31 624
436 Flume과 Kafka를 사용한 초당 100만개 로그 수집 테스트 file 총관리자 2016.10.31 1021
435 Spark Streaming 코드레벨단에서의 성능개선 총관리자 2016.10.31 44
434 centos 6에서 mariadb 5.1 to 10.0 으로 upgrade 총관리자 2016.11.01 111
433 java스레드 덤프 분석하기 file 총관리자 2016.11.03 111
432 데이타 분석및 머신러닝에 도움이 도움이 되는 사이트 총관리자 2016.11.04 64
431 [SparkR]SparkR 설치 사용기 1 - Installation Guide On Yarn Cluster & Mesos Cluster & Stand Alone Cluster file 총관리자 2016.11.04 106
430 Eclipse실행시 Java was started but returned exit code=1이라는 오류가 발생할때 조치방법 총관리자 2016.11.07 398
429 Kafka Offset Monitor로 kafka 상태 모니터링 하기 file 총관리자 2016.11.08 529
428 참고할만한 spark예제를 설명하는 사이트 총관리자 2016.11.11 98
427 spark notebook 0.7.0설치및 설정 총관리자 2016.11.14 160
426 git 초기화(Windows에서 Git Bash사용) 총관리자 2016.11.17 197
425 특정 단계의 commit상태로 만들기(이렇게 하면 중간에 반영된 모든 commit를 history가 삭제된다) 총관리자 2016.11.17 45
424 Github를 이용하는 전체 흐름 이해하기 총관리자 2016.11.18 36
423 특정 커밋 시점(commit id를 기준으로)으로 돌리기(reset) 총관리자 2016.11.21 75
422 .gitignore파일에 지정되지 않은 파일이 ignore되는 경우 확인방법 총관리자 2016.11.22 107
쓰기 태그

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

위로