메뉴 건너뛰기

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.


hadoop클러스터를 구성 하던 서버중 HA를 담당하는 서버의 hostname등이 변경되었을때는 "hadoop-daemon.sh start zkfc"를 수행할때

아래와 같은 오류가 발생할 수 있는데 zookeeper내의 /hadoop-ha/mycluster 노드에 있는 정보가 변경된 사항을 반영하지 못해서 문제가

발생한것이다.


이때는 "hdfs zkfc -formatZK"를 실행하여 ZKFC정보를 재생성해준다.


-----------------hdfs zkfc -formatZK실행 로그----------------

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:java.library.path=:/svc/apps/sda/bin/hadoop/hadoop/lib/native

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-573.22.1.el6.x86_64

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:user.name=root

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:user.home=/root

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/gooper/svc/apps/sda/bin/hadoop/hadoop-2.7.2/bin

16/07/29 19:30:43 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=gsda1:2181,gsda2:2181,gsda3:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@1e178745

16/07/29 19:30:43 INFO zookeeper.ClientCnxn: Opening socket connection to server sda2/XXX.XXX.XXX.44:2181. Will not attempt to authenticate using SASL (unknown error)

16/07/29 19:30:43 INFO zookeeper.ClientCnxn: Socket connection established to sda2/XXX.XXX.XXX.44:2181, initiating session

16/07/29 19:30:43 INFO zookeeper.ClientCnxn: Session establishment complete on server sda2/XXX.XXX.XXX.44:2181, sessionid = 0x25634f0fabb0300, negotiated timeout = 5000

16/07/29 19:30:43 INFO ha.ActiveStandbyElector: Session connected.

===============================================

The configured parent znode /hadoop-ha/mycluster already exists.

Are you sure you want to clear all failover information from

ZooKeeper?

WARNING: Before proceeding, ensure that all HDFS services and

failover controllers are stopped!

===============================================

Proceed formatting /hadoop-ha/mycluster? (Y or N) Y

16/07/29 19:36:25 INFO ha.ActiveStandbyElector: Recursively deleting /hadoop-ha/mycluster from ZK...

16/07/29 19:36:25 INFO ha.ActiveStandbyElector: Successfully deleted /hadoop-ha/mycluster from ZK.

16/07/29 19:36:26 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/mycluster in ZK.

16/07/29 19:36:26 INFO zookeeper.ZooKeeper: Session: 0x25634f0fabb0300 closed

16/07/29 19:36:26 INFO zookeeper.ClientCnxn: EventThread shut down



------------------오류내용-------------

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/home/gooper/svc/apps/sda/bin/hadoop/hadoop-2.7.2/lib

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-573.12.1.el6.x86_64

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=root

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/root

2016-07-29 18:33:32,857 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/gooper/svc/apps/sda/bin/hadoop/hadoop-2.7.2

2016-07-29 18:33:32,858 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=gsda1:2181,gsda2:2181,gsda3:2181 sessionTimeout=5000 watcher=org.apache.hadoop.ha.Ac

tiveStandbyElector$WatcherWithClientRef@52bf72b5

2016-07-29 18:33:32,936 FATAL org.apache.hadoop.hdfs.tools.DFSZKFailoverController: Got a fatal error, exiting now

java.net.UnknownHostException: gsda3: unknown error

        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)

        at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)

        at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)

        at java.net.InetAddress.getAllByName0(InetAddress.java:1276)

        at java.net.InetAddress.getAllByName(InetAddress.java:1192)

        at java.net.InetAddress.getAllByName(InetAddress.java:1126)

        at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:61)

        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)

        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)

        at org.apache.hadoop.ha.ActiveStandbyElector.getNewZooKeeper(ActiveStandbyElector.java:631)

        at org.apache.hadoop.ha.ActiveStandbyElector.createConnection(ActiveStandbyElector.java:775)

        at org.apache.hadoop.ha.ActiveStandbyElector.<init>(ActiveStandbyElector.java:229)

        at org.apache.hadoop.ha.ZKFailoverController.initZK(ZKFailoverController.java:350)

        at org.apache.hadoop.ha.ZKFailoverController.doRun(ZKFailoverController.java:191)

        at org.apache.hadoop.ha.ZKFailoverController.access$000(ZKFailoverController.java:61)

        at org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:172)

        at org.apache.hadoop.ha.ZKFailoverController$1.run(ZKFailoverController.java:168)

        at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:415)

        at org.apache.hadoop.ha.ZKFailoverController.run(ZKFailoverController.java:168)

        at org.apache.hadoop.hdfs.tools.DFSZKFailoverController.main(DFSZKFailoverController.java:181)

번호 제목 글쓴이 날짜 조회 수
257 [SBT] assembly시 "[error] deduplicate: different file contents found in the following:"오류 발생시 조치사항 총관리자 2016.08.04 575
256 [SBT] SBT 사용법 정리(링크) 총관리자 2016.08.04 838
255 [SBT] project.sbt에 libraryDependencies에 필요한 jar를 지정했으나 sbt compile할때 클래스를 못찾는 오류가 발생했을때 조치사항 총관리자 2016.08.03 74
254 build할때 unmappable character for encoding MS949 에러 발생시 조치사항 총관리자 2016.08.03 178
253 kafkaWordCount.scala의 producer와 consumer 클래스를 이용하여 kafka를 이용한 word count 테스트 하기 총관리자 2016.08.02 97
252 bin/start-hbase.sh실행시 org.apache.hadoop.hbase.util.FileSystemVersionException: HBase file layout needs to be upgraded오류가 발생하면 조치사항 총관리자 2016.08.01 205
251 start-all.sh로 spark데몬 기동시 "JAVA_HOME is not set"오류 발생시 조치사항 총관리자 2016.08.01 126
» hadoop클러스터를 구성하던 서버중 HA를 담당하는 서버의 hostname등이 변경되어 문제가 발생했을때 조치사항 총관리자 2016.07.29 362
249 Journal Storage Directory /data/hadoop/journal/data/mycluster not formatted 오류시 조치사항 총관리자 2016.07.29 1518
248 슬라이딩 윈도우 예제 총관리자 2016.07.28 67
247 거침없이 배우는 Drools 책의 샘플소스 file 총관리자 2016.07.22 1232
246 drools를 이용한 로그,rule matching등의 테스트 java프로그램 file 총관리자 2016.07.21 181
245 ServerInfo객체파일 총관리자 2016.07.21 35
244 drools에서 drl관련 로그를 기록하기 위한 클래스 파일 총관리자 2016.07.21 74
243 워킹 메모리에 대한 정보를 처리하는 클래스 파일 총관리자 2016.07.21 49
242 커리 변경 이벤트를 처리하기 위한 구현클래스 총관리자 2016.07.21 41
241 룰에 매칭되면 발생되는 엑티베이션 객체에 대한 작업(이전값 혹은 현재값)을 처리하는 클래스 파일 총관리자 2016.07.21 285
240 실시간 쿼리 변환 모니터링(팩트내 필드값의 변경사항을 실시간으로 추적함)하는 테스트 java 프로그램 file 총관리자 2016.07.21 67
239 Drools 6.0 - 비즈니스 룰 기반으로 간단한 룰 애플리케이션 만들기 file 총관리자 2016.07.18 434
238 Apache Spark와 Drools를 이용한 CEP구현 테스트 총관리자 2016.07.15 342

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

위로