메뉴 건너뛰기

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.


1. microservice pattern책의 부록 부분을 참고하여 window에서 리눅스를 사용할 수 있도록(WSL2등) 설치/설정한다.


2. kubernetes로 서비스를 기동할때 몇가지 수정해야할 부분이 있다. 

  (kubernetes가 버젼이 올라가면서 예제로 만들어진 kubernetes기동용 yaml과 script자체가 잘못되거나 없어진 옵션을 사용하고 있어 몇가지를 수정해야한다.)

  가. apps/v1beta1를 apps/v1로 수정

  나. extentions/v1beta1를 apps/v1로 수정

  다. selector.matchLabels.app에 app이름 추가

  라. metadata의 label항목에 app: app이름 추가


3. github소스 받아 build및 docker이미 생성

가. git clone github.com/microservice-patterns/ftgo-application

나. cd ftgo-application

다. ./gradlew buildContracts

라. ./gradlew assemble


4. container 배포

가. cd ~/ftgo-application/deployment/kubernetes/scripts

나. ./kubernetes-deploy-all.sh 

gooper@DESKTOP-PJ7F3IU:~/ftgo-application/deployment/kubernetes/scripts$ ./kubernetes-deploy-all.sh

secret/ftgo-db-secret created

service/ftgo-dynamodb-local created

statefulset.apps/ftgo-dynamodb-local created

job.batch/ftgo-dynamodb-local-init created

service/ftgo-kafka created

statefulset.apps/ftgo-kafka created

service/ftgo-mysql created

statefulset.apps/ftgo-mysql created

service/ftgo-zookeeper created

statefulset.apps/ftgo-zookeeper created

....................................................................................................................................connected

service/ftgo-cdc-service created

deployment.apps/ftgo-cdc-service created


5. container삭제

gooper@DESKTOP-PJ7F3IU:~/ftgo-application/deployment/kubernetes/scripts$ ./kubernetes-delete-all.sh

service "ftgo-cdc-service" deleted

deployment.apps "ftgo-cdc-service" deleted

secret "ftgo-db-secret" deleted

service "ftgo-dynamodb-local" deleted

statefulset.apps "ftgo-dynamodb-local" deleted

job.batch "ftgo-dynamodb-local-init" deleted

service "ftgo-kafka" deleted

statefulset.apps "ftgo-kafka" deleted

service "ftgo-mysql" deleted

statefulset.apps "ftgo-mysql" deleted

service "ftgo-zookeeper" deleted

statefulset.apps "ftgo-zookeeper" deleted


6. port forwarding하기 (이거 반드시 실행해줘야 한다)

가. cd ~/ftgo-application/deployment/kubernetes/scripts

나. ./port-forwards.sh


7. swagger ui를 이용한 서비스 테스트

가. consumer controller; http://127.0.0.1:8081/swagger-ui/

나. order controller, restaurant conroller; http://127.0.0.1:8082/swagger-ui/

다. kitchen controller, restaurant conroller; http://127.0.0.1:8083/swagger-ui/

라. basic error controller, operation handler, restaurant conrolle, web-mvc-link-handler ; http://127.0.0.1:8084/swagger-ui/

마. accounts controller; http://127.0.0.1:8085/swagger-ui/


* 수정해야 하는 파일 #1

gooper@DESKTOP-PJ7F3IU:~/ftgo-application/deployment/kubernetes/scripts$ ls -al ~/ftgo-application/*/src/deployment/kubernetes/*.yml

-rw-r--r-- 1 gooper gooper 1820 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-accounting-service/src/deployment/kubernetes/ftgo-accounting-service.yml

-rw-r--r-- 1 gooper gooper 1542 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-api-gateway/src/deployment/kubernetes/ftgo-api-gateway.yml

-rw-r--r-- 1 gooper gooper 1780 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-consumer-service/src/deployment/kubernetes/ftgo-consumer-service.yml

-rw-r--r-- 1 gooper gooper 1802 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-kitchen-service/src/deployment/kubernetes/ftgo-kitchen-service.yml

-rw-r--r-- 1 gooper gooper 2123 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-order-history-service/src/deployment/kubernetes/ftgo-order-history-service.yml

-rw-r--r-- 1 gooper gooper 1931 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-order-service/src/deployment/kubernetes/ftgo-order-service.yml

-rw-r--r-- 1 gooper gooper 1853 Jan 30 10:17 /home/gooper/ftgo-application/ftgo-restaurant-service/src/deployment/kubernetes/ftgo-restaurant-service.yml


* 수정해야 하는 파일 #2
gooper@DESKTOP-PJ7F3IU:~/ftgo-application/deployment/kubernetes/stateful-services$ ls -al
total 28
drwxr-xr-x 2 gooper gooper 4096 Jan 30 13:28 .
drwxr-xr-x 6 gooper gooper 4096 Jan 30 14:30 ..
-rw-r--r-- 1 gooper gooper  138 Jan 30 10:17 ftgo-db-secret.yml
-rw-r--r-- 1 gooper gooper 1651 Jan 30 13:21 ftgo-dynamodb-local.yml
-rw-r--r-- 1 gooper gooper 1215 Jan 30 13:26 ftgo-kafka-deployment.yml
-rw-r--r-- 1 gooper gooper 1250 Jan 30 13:24 ftgo-mysql-deployment.yml
-rw-r--r-- 1 gooper gooper 1081 Jan 30 13:25 ftgo-zookeeper-deployment.yml

* 수정해야 하는 파일 #3
~/ftgo-application/deployment/kubernetes/cdc-service/ftgo-cdc-service.yml

* 수정해야 하는 파일 #4(typo및 경로를 수정해야함)
gooper@DESKTOP-PJ7F3IU:~/ftgo-application/deployment/kubernetes/scripts$ ls -al *.sh
-rwxr-xr-x 1 gooper gooper 219 Jan 30 15:01 kubernetes-delete-all.sh
-rwxr-xr-x 1 gooper gooper 231 Jan 30 10:17 kubernetes-delete-volumes.sh
-rwxr-xr-x 1 gooper gooper 450 Jan 30 14:26 kubernetes-deploy-all.sh
-rwxr-xr-x 1 gooper gooper 156 Jan 30 10:17 kubernetes-deploy-and-test.sh
-rwxr-xr-x 1 gooper gooper  76 Jan 30 10:17 kubernetes-kill-port-forwarding.sh
-rwxr-xr-x 1 gooper gooper 316 Jan 30 10:17 kubernetes-run-end-to-end-tests.sh
-rwxr-xr-x 1 gooper gooper 513 Jan 30 10:17 kubernetes-wait-for-ready-pods.sh
-rwxr-xr-x 1 gooper gooper 621 Jan 30 10:17 port-forwards.sh

번호 제목 글쓴이 날짜 조회 수
699 org.apache.hadoop.security.AccessControlException: Permission denied: user=hadoop, access=WRITE, inode="":root:supergroup:rwxr-xr-x 오류 처리방법 총관리자 2014.07.05 2833
698 spark-sql실행시 Caused by: java.lang.NumberFormatException: For input string: "0s" 오류발생시 조치사항 총관리자 2016.06.09 2802
697 banana pi에(lubuntu)에 hadoop설치하고 테스트하기 - 성공 file 총관리자 2014.07.05 2760
696 protege 설명및 사용법 file 총관리자 2017.04.04 2731
695 Hive+mysql 설치 및 환경구축하기 file 구퍼 2013.03.07 2722
694 org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 구퍼 2013.03.15 2668
693 mysql-server 기동시 Do you already have another mysqld server running on port 오류 발생할때 확인및 조치방법 총관리자 2017.05.14 2664
692 HBase 설치하기 – Pseudo-distributed file 구퍼 2013.03.12 2644
691 hadoop 설치(3대) file 구퍼 2013.03.07 2613
690 HBase, BigTable, Cassandra Schema Design file 구퍼 2013.03.15 2506
689 hbase shell에서 컬럼값 검색하기(SingleColumnValueFilter이용) 총관리자 2014.04.25 2446
688 banana pi에 hive 0.13.1+mysql(metastore)설치 file 총관리자 2014.09.09 2406
687 Cacti로 Hadoop 모니터링 하기 file 구퍼 2013.03.12 2367
686 AIX 7.1에 MariaDB 10.2 소스 설치 총관리자 2016.09.24 2361
685 jupyter, zeppelin, rstudio를 이용하여 spark cluster에 job를 실행시키기 위한 정보 총관리자 2018.04.13 2335
684 kafka broker기동시 brokerId가 달라서 기동에 실패하는 경우 조치방법 총관리자 2016.05.02 2328
683 hadoop설치시 오류 총관리자 2013.12.18 2313
682 메이븐 (maven) 설치 및 이클립스 연동하기 file 구퍼 2013.03.06 2280
681 apk 파일 위치 file 총관리자 2015.05.25 2227
680 hadoop설치시 참고사항 구퍼 2013.03.08 2131

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

위로