메뉴 건너뛰기

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.


CDH설치중 아래와 같이 오류가 발생하면서 cloudera agent가 설치되지 않는 경우가 있는데 이는 apt-get update를 실행하면서 package "apt-transport-https"가 설치되지 않아서 발생하는 문제이므로 sudo apt-get install apt-transport-https를 실행후 agent설치를 retry해준다.


----------sudo apt-get update를 실행하여 보여지는 에러 메세지--------------

gooper@gsda3:/etc/apt/sources.list.d$ sudo apt-get update

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease

Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease                                        

Hit:3 http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease                        

Hit:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease                                    

Hit:5 http://ftp.kaist.ac.kr/mariadb/repo/10.1/ubuntu xenial InRelease  

Reading package lists... Done

E: The method driver /usr/lib/apt/methods/https could not be found.

N: Is the package apt-transport-https installed?

E: Failed to fetch https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.14.2/InRelease  

E: Some index files failed to download. They have been ignored, or old ones used instead.


-------apt-transport-https패키지를 설치하는 메세지-------------

gooper@gsda3:/etc/apt/sources.list.d$ sudo apt-get install apt-transport-https

Reading package lists... Done

Building dependency tree       

Reading state information... Done

The following package was automatically installed and is no longer required:

  postgresql

Use 'sudo apt autoremove' to remove it.

The following NEW packages will be installed:

  apt-transport-https

0 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.

Need to get 26.1 kB of archives.

After this operation, 215 kB of additional disk space will be used.

Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-transport-https amd64 1.2.26 [26.1 kB]

Fetched 26.1 kB in 3s (7,659 B/s)                    

Selecting previously unselected package apt-transport-https.

(Reading database ... 35334 files and directories currently installed.)

Preparing to unpack .../apt-transport-https_1.2.26_amd64.deb ...

Unpacking apt-transport-https (1.2.26) ...

Setting up apt-transport-https (1.2.26) ...


-----------------------CM설치하면서 보여지는 오류 메세지-----------------

/tmp/scm_prepare_node.DD6Kh9k4 

using SSH_CLIENT to get the SCM hostname: 104.251.212.146 57750 10022 

opening logging file descriptor 

Starting installation script...

Acquiring installation lock...

BEGIN flock 4 

END (0) 

Detecting root privileges...

effective UID is 1001 

BEGIN which pbrun 

END (1) 

BEGIN sudo -S id 

uid=0(root) gid=0(root) groups=0(root) 

END (0) 

Using 'sudo ' to acquire root privileges 

Detecting distribution...

BEGIN grep 'Ubuntu' /etc/lsb-release 

DISTRIB_ID=Ubuntu 

DISTRIB_DESCRIPTION="Ubuntu 16.04.4 LTS" 

END (0) 

BEGIN grep DISTRIB_CODENAME /etc/lsb-release 

END (0) 

DISTRIB_CODENAME=xenial 

BEGIN echo DISTRIB_CODENAME=xenial | cut -d = -f 2 

xenial 

END (0) 

Detecting Cloudera Manager Server...

BEGIN host -t PTR 104.251.212.146 

/tmp/scm_prepare_node.DD6Kh9k4/scm_prepare_node.sh: line 104: host: command not found 

END (127) 

BEGIN which python 

/usr/bin/python 

END (0) 

BEGIN python -c 'import socket; import sys; s = socket.socket(socket.AF_INET); s.settimeout(5.0); s.connect((sys.argv[1], int(sys.argv[2]))); s.close();' 104.251.212.146 7182 

END (0) 

BEGIN which wget 

END (0) 

/usr/bin/wget 

BEGIN wget -qO- -T 1 -t 1 http://169.254.169.254/latest/meta-data/public-hostname && /bin/echo 

END (4) 

Installing package repositories...

validating format of repository file /tmp/scm_prepare_node.DD6Kh9k4/repos/ubuntu_xenial/cloudera-manager.list 

installing repository file /tmp/scm_prepare_node.DD6Kh9k4/repos/ubuntu_xenial/cloudera-manager.list 

repository file /tmp/scm_prepare_node.DD6Kh9k4/repos/ubuntu_xenial/cloudera-manager.list installed 

BEGIN sudo apt-key add /tmp/scm_prepare_node.DD6Kh9k4/archive.key 

OK 

END (0) 

installing priority file /tmp/scm_prepare_node.DD6Kh9k4/priorities/ubuntu_xenial/cloudera.pref 

priority file /tmp/scm_prepare_node.DD6Kh9k4/priorities/ubuntu_xenial/cloudera.pref installed 

Refreshing package metadata...

BEGIN sudo apt-get update 

Hit:1 http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease 

Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease 

Hit:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease 

Hit:4 http://ftp.kaist.ac.kr/mariadb/repo/10.1/ubuntu xenial InRelease 

Hit:5 http://security.ubuntu.com/ubuntu xenial-security InRelease 

Reading package lists... 

E: The method driver /usr/lib/apt/methods/https could not be found. 

E: Failed to fetch https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.14.2/InRelease 

E: Some index files failed to download. They have been ignored, or old ones used instead. 

END (100) 

BEGIN sudo apt-get update 

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease 

Hit:2 http://ppa.launchpad.net/webupd8team/java/ubuntu xenial InRelease 

Hit:3 http://archive.ubuntu.com/ubuntu xenial InRelease 

Hit:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease 

Hit:5 http://ftp.kaist.ac.kr/mariadb/repo/10.1/ubuntu xenial InRelease 

Reading package lists... 

E: The method driver /usr/lib/apt/methods/https could not be found. 

E: Failed to fetch https://archive.cloudera.com/cm5/ubuntu/xenial/amd64/cm/dists/xenial-cm5.14.2/InRelease 

E: Some index files failed to download. They have been ignored, or old ones used instead. 

END (100) 

Installing cloudera-manager-agent package...

BEGIN sudo dpkg -l cloudera-manager-agent | grep -E '^ii[[:space:]]*cloudera-manager-agent[[:space:]]*' 

dpkg-query: no packages found matching cloudera-manager-agent 

END (1) 

BEGIN sudo apt-cache show cloudera-manager-agent 

E: No packages found 

END (100) 

cloudera-manager-agent must have Version=5.14.2 and Build=8, exiting


번호 제목 글쓴이 날짜 조회 수
378 [AD(LADP)] CDP1.7에서 AD및 Kerberos를 연동해도 각 노드에 os account, os group은 생성되어야 하지만 SSSD서비스를 이용하면 직접 생성될 필요가 없다. gooper 2022.06.10 105
377 Query 1234:1234 expired due to client inactivity(timeout is 5m)및 invalid query handle gooper 2022.06.10 81
376 HDFS 파일및 디렉토리 생성시 생성방법에 따라 권한이 다르게 부여된다. gooper 2022.05.30 217
375 [Cloudera Agent] Metadata-Plugin throttling_logger INFO (713 skipped) Unable to send data to nav server. Will try again. gooper 2022.05.16 7
374 "bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')])" 오류는 CA인증을 하지 못해서 발생함 총관리자 2022.05.13 48
373 [TLS/SSL]Kudu Tablet Server설정 총관리자 2022.05.13 35
372 [TLS/SSL]Kudu Master 설정하기 총관리자 2022.05.13 61
371 [TLS/SSL]Cloudera 6.3.4기준 Oozie Web UI TLS설정 항목및 설정값 총관리자 2022.05.13 36
370 [TLS/SSL]Cloudera CDH6.3.4기준 Hue TLS설정 항목 총관리자 2022.05.13 93
369 [HIVESERVER2]프로세스의 thread및 stack trace를 덤프하는 방법(pstack, jstack) 총관리자 2022.05.11 149
368 [Hue]Hue의 메타정보를 담고 있는 desktop_document테이블과 desktop_document2의 관계 총관리자 2022.05.09 38
367 Error: IO_ERROR : java.io.IOException: Error while connecting Oozie server 총관리자 2022.05.02 26
366 oozie webui접근시 id/pw를 물어보는 Windows보안 팝업창이 뜰때 확인/조치방법 총관리자 2022.05.02 19
365 Could not authenticate, GSSException: No valid credentials provided (Mechanism level: Failed to find any kerberos tgt) 총관리자 2022.04.28 27
364 [hive] hive.tbls테이블의 owner컬럼값은 hadoop.security.auth_to_local에 의해서 filtering된다. 총관리자 2022.04.14 55
363 hue메타 정보를 저장(oracle DB)하는 내부 테이블을 이용하여 전체 테이블목록, 전체 코디네이터 목록, 코디네이터기준 workflow구조를 추출하는 쿼리문 총관리자 2022.04.01 48
362 HDFS에서 quota 설정 방법및 확인 방법 총관리자 2022.03.30 55
361 [oozie]Oozie WF수행시 단계별 ID넘버링 비교/설명 총관리자 2022.03.23 16
360 [application수행 로그]Failed to read the application application_123456789012_123456시 조치 방법 총관리자 2022.03.21 46
359 Hue impala에서 query결과를 HDFS 파일로 export시 AuthorizationException: User 'gooper1234' does not have privileges to access: db명.query_impala_123456 총관리자 2022.03.17 214

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

위로