Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.

spark spark2.0.0에서 hive 2.0.1 table을 읽어 출력하는 예제 소스(HiveContext, SparkSession, SQLContext)

총관리자 2017.03.09 14:57 조회 수 : 163

* 주의 : sQLContext, HiveContext는 "The type HiveContext is deprecated" 상태임

* 주의 : HiveContext는 window함수를 사용할수 있지만(예, SELECT value, dense_rank() OVER (PARTITION BY key ORDER BY value DESC) as rank FROM src) SQLContext는 사용할 수 없음

1. hive table 생성및 데이타 로드, 확인

hive> create table test_table(name varchar(100), age int)

row format delimited fields terminated by ','

lines terminated by 'n';

hive> load data local inpath '/tmp/test.txt' into table test_table;

hive> select * from test_table;
OK
park 10
sang 20
hyeon 30

2. spark에서 hive테이블을 다루기 위한 dependency설정

가. pom.xml파일에 dependency추가

<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-hive_2.11</artifactId> <version>2.0.0</version> <scope>provided</scope> </dependency>

3. 여러가지 경우의 샘플소스

가. 테스트 메서드를 호출하는 부분

public static void main() {

SparkConf sparkConf=new SparkConf().setAppName("JavaSparkTest");

JavaSparkContext jsc=new JavaSparkContext(sparkConf);

test14(jsc);

}

1) HiveContext를 이용하는 경우
static void test14(JavaSparkContext sc) {
   HiveContext ctx = new HiveContext(sc);
   Dataset<Row> rows = ctx.sql("select * from default.test_table");

   Dataset<String> stringsDS = rows.map(new MapFunction<Row, String>() {
    @Override
    public String call(Row row) throws Exception {
     //return "Key : "+row.get(0) + ", Value : "+row.get(1);
     return "Value : "+row.get(0);
    }
   }, Encoders.STRING());

   System.out.println("result #1 =>"+stringsDS.toJavaRDD().collect());
}
---> 실행결과
result #1 =>[Value : park, Value : sang, Value : hyeon]

2) SparkSession을 이용하는 경우
static void test15() {
   SparkSession session = new SparkSession.Builder().appName("SparkJoinExample").master("local").enableHiveSupport().getOrCreate();
      Dataset<Row> dset = session.sql("select * from default.test_table");

   System.out.println("result #2 =>"+dset.toJavaRDD().collect());
}
--->실행결과
result #2 =>[[park], [sang], [hyeon]]

3) SQLContext를 이용하는 경우
static void test16(JavaSparkContext sc) {
   SQLContext ctx = new SQLContext(sc);
   Dataset<Row> rows = ctx.sql("select * from default.test_table");

   Dataset<String> stringsDS = rows.map(new MapFunction<Row, String>() {
    @Override
    public String call(Row row) throws Exception {
     //return "Key : "+row.get(0) + ", Value : "+row.get(1);
     return "Value : "+row.get(0);
    }
   }, Encoders.STRING());

   System.out.println("result #3 =>"+stringsDS.toJavaRDD().collect());
}
--->실행결과
result #3 =>[Value : park, Value : sang, Value : hyeon]

--------------------HiveContext를 사용할때 아래와 같은 경고및 에러 메세지가 발생하지만.. 결과는 나온다...-----

[2017-03-09 18:35:28,674] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:28,675] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:28,676] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:28,677] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:28,678] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:28,679] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:28,680] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:28,681] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:28,682] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:28,683] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:28,684] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:28,685] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:28,686] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:28,687] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:28,688] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:28,689] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:29,205] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:29,205] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:29,206] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:29,207] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:29,208] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:29,209] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:29,210] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:29,211] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:29,212] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:29,213] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:29,214] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:29,215] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:29,216] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:29,217] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:29,218] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:29,219] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:29,220] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.false.positive.probability does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.orc.time.counters does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.metrics.enabled does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.timeout.ms does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.connect.retry.limit does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.direct does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.stats does not exist 
[2017-03-09 18:35:30,057] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.client.consistent.splits does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.start does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.ttl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.acl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.delegation.token.lifetime does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.large.query does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.min does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.user does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.wait.queue.comparator.class.name does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.am.liveness.heartbeat.interval.ms does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.object.cache.enabled does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.ssl does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.location does not exist 
[2017-03-09 18:35:30,058] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.delay.seconds does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.file.cleaner.threads does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.fail.compaction does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.class does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.vectorized.execution.reducesink.new.enabled does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.max.num.delta does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.attempted does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.port does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.initiator.failed.compacts.threshold does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.reporter does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.execution.mode does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.mode does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.threadpool.size does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.use.spnego does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.service.metrics.file.frequency does not exist 
[2017-03-09 18:35:30,059] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.fastpath does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.file.cleanup.delay.seconds does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.management.rpc.port does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.tree does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.failed does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.host does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.ms.footer.cache.enabled does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup.min does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.service.refresh.interval.sec does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.output.size does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.driver.parallel.compilation does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.cache.allow.synthetic.fileid does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.hash.table.inflation.factor does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.hbase.ttl does not exist 
[2017-03-09 18:35:30,060] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.enforce.vectorized does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.exec.schema.evolution does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.llap.concurrent.queries does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.allow.uber does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.auth does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.include.fileid does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.communicator.num.threads does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.partitions does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.shuffle.port does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.response.header.size does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.internal.variable.list does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductionpercentage does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.enabled does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.retry.limit does not exist 
[2017-03-09 18:35:30,061] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.orc.splits.directory.batch.ms does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.reader.wait does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.max.timeout.ms does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.execution.mode does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.catalog.cache.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.wait.queue.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.cache.entries does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.memory.ttl does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.port does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.cache.size does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.clean.until does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.port does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.metrics.enabled does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.cli.tez.session.async does not exist 
[2017-03-09 18:35:30,062] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.bucket.pruning.compat does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.spnego.principal does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.shuffle.dir.watcher.enabled does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.arena.count does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.communicator.connection.timeout.ms does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.transpose.aggr.join does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.spark.dynamic.partition.pruning.max.data.size does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggr.stats.invalidator.frequency does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.lrfu does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.conf.hidden.list does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.timedout.txn.reaper.interval does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.aggregate.stats.max.variance does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.lrfu.lambda does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.point.lookup does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.ssl does not exist 
[2017-03-09 18:35:30,063] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.retention.succeeded does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.use.fileid.path does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.mapjoin.optimized.hashtable.probe.percent does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.reenable.min.timeout.ms does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.support.special.characters.tablename does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.yarn.container.mb does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.dynamic.partition.hashjoin does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.task.scheduler.enable.preemption does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.num.executors does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.full does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.connection.class does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.client.password does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.metastore.hbase.cache.max.writer.wait does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.thrift.http.request.header.size does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.webui.max.threads does not exist 
[2017-03-09 18:35:30,064] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.limittranspose.reductiontuples does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.test.rollbacktxn does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.num.schedulable.tasks.per.node does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.acl does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.memory.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.type.safety does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.auto.max.input.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.enable.memory.manager does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.server2.tez.session.lifetime.jitter does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.web.port does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.strict.checks.cartesian.product does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.rpc.num.handlers does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.vcpus.per.instance does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.optimize.partition.columns.separate does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.txn.heartbeat.threadpool.size does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.locality.delay does not exist 
[2017-03-09 18:35:30,065] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.task.scheduler.node.disable.backoff.factor does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.am.liveness.connection.sleep.between.retries.ms does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.daemon.memory.per.instance.mb does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.msck.path.validation does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.compactor.history.reaper.interval does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.tez.input.generate.consistent.splits does not exist 
[2017-03-09 18:35:30,066] [conf.HiveConf] [initialize(#2753)] [WARN] HiveConf of name hive.llap.io.allocator.alloc.max does not exist 
[2017-03-09 18:35:30,239] [metastore.RetryingHMSHandler] [invoke(#159)] [ERROR] AlreadyExistsException(message:Database default already exists)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_database(HiveMetaStore.java:891)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at com.sun.proxy.$Proxy28.create_database(Unknown Source)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:644)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156)
        at com.sun.proxy.$Proxy29.createDatabase(Unknown Source)
        at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:306)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply$mcV$sp(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$createDatabase$1.apply(HiveClientImpl.scala:291)
        at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:262)
        at org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:209)
        at org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:208)
        at org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:251)
        at org.apache.spark.sql.hive.client.HiveClientImpl.createDatabase(HiveClientImpl.scala:290)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply$mcV$sp(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$createDatabase$1.apply(HiveExternalCatalog.scala:99)
        at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:72)
        at org.apache.spark.sql.hive.HiveExternalCatalog.createDatabase(HiveExternalCatalog.scala:98)
        at org.apache.spark.sql.catalyst.catalog.SessionCatalog.createDatabase(SessionCatalog.scala:147)
        at org.apache.spark.sql.catalyst.catalog.SessionCatalog.<init>(SessionCatalog.scala:89)
        at org.apache.spark.sql.hive.HiveSessionCatalog.<init>(HiveSessionCatalog.scala:51)
        at org.apache.spark.sql.hive.HiveSessionState.catalog$lzycompute(HiveSessionState.scala:49)
        at org.apache.spark.sql.hive.HiveSessionState.catalog(HiveSessionState.scala:48)
        at org.apache.spark.sql.hive.HiveSessionState$$anon$1.<init>(HiveSessionState.scala:63)
        at org.apache.spark.sql.hive.HiveSessionState.analyzer$lzycompute(HiveSessionState.scala:63)
        at org.apache.spark.sql.hive.HiveSessionState.analyzer(HiveSessionState.scala:62)
        at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:49)
        at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:64)
        at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:582)
        at org.apache.spark.sql.SQLContext.sql(SQLContext.scala:682)
        at com.gooper.icbms.sda.kafka.onem2m.JavaSparkPi.test14(JavaSparkPi.java:523)
        at com.gooper.icbms.sda.kafka.onem2m.JavaSparkPi.main(JavaSparkPi.java:165)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:729)
        at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

이 게시물을

번호	제목	글쓴이	날짜	조회 수
21	spark 2.0.0의 api를 이용하는 예제 프로그램	총관리자	2017.03.15	199
20	Scala에서 countByWindow를 이용하기(예제)	총관리자	2018.03.08	235
19	Windows7 64bit 환경에서 Apache Spark 2.2.0 설치하기	총관리자	2017.07.26	260
18	Caused by: java.lang.ClassNotFoundException: org.apache.spark.Logging 발생시 조치사항	총관리자	2017.04.19	284
17	Spark에서 KafkaUtils.createStream()를 이용하여 이용하여 kafka topic에 접근하여 객채로 저장된 값을 가져오고 처리하는 예제 소스	총관리자	2017.04.26	292
16	spark-submit으로 spark application실행하는 다양한 방법	총관리자	2016.05.25	303
15	Apache Spark와 Drools를 이용한 CEP구현 테스트	총관리자	2016.07.15	342
14	Spark 1.6.1 설치후 HA구성	총관리자	2016.05.24	455
13	spark-sql실행시 The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH오류 발생시 조치사항	총관리자	2016.06.09	455
12	java.lang.OutOfMemoryError: unable to create new native thread오류 발생지 조치사항	총관리자	2016.10.17	469
11	spark-submit 실행시 "java.lang.OutOfMemoryError: Java heap space"발생시 조치사항	총관리자	2018.02.01	517
10	spark client프로그램 기동시 "Error initializing SparkContext"오류 발생할때 조치사항	총관리자	2016.05.27	539
9	spark-shell실행시 "A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection."오류가 발생하는 경우 해결방법	총관리자	2016.05.20	551
8	spark-env.sh에서 사용할 수있는 항목.	총관리자	2016.05.24	567
7	kafka로 부터 메세지를 stream으로 받아 처리하는 spark샘플소스(spark의 producer와 consumer를 sbt로 컴파일 하고 서버에서 spark-submit하는 방법)	총관리자	2016.07.13	630
6	"Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources"오류 발생시 조치사항	총관리자	2016.05.25	1030
5	spark stream처리할때 두개의 client프로그램이 동일한 checkpoint로 접근할때 발생하는 오류 내용	총관리자	2018.01.16	1115
4	Spark 2.1.1 clustering(5대) 설치(YARN기반)	총관리자	2016.04.22	1882
3	VisualVM 1.3.9을 이용한 spark-submit JVM 모니터링을 위한 설정및 spark-submit실행 옵션	총관리자	2016.10.28	1891
2	spark-sql실행시 Caused by: java.lang.NumberFormatException: For input string: "0s" 오류발생시 조치사항	총관리자	2016.06.09	2802

쓰기 태그

첫 페이지 1 2 3 끝 페이지

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.

spark spark2.0.0에서 hive 2.0.1 table을 읽어 출력하는 예제 소스(HiveContext, SparkSession, SQLContext)

댓글 0

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다. 필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.

spark spark2.0.0에서 hive 2.0.1 table을 읽어 출력하는 예제 소스(HiveContext, SparkSession, SQLContext)

댓글 0

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc. We are open to the required minutes. Please send inquiries to gooper@gooper.com.

LOGIN

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.