메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


1. 현재 연결된 기기 확인

(아래는 31.7GB(/dev/mmcblk0)과 1000.2 GB(/dev/sda)의 2개의 외장기기 확인됨

   /dev/mmcblk0는 /dev/mmcblk0p1과 /dev/mmcblk0p2의 2개 파티션으로 구성 되어 있고

          /dev/sda는 /dev/sda1로 파티션 1개로 구성되어 있음)

root@slave1:~# fdisk -l

Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes
4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xf7e9dfe5

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux
/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xc197d009

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048  1953521663   976759808    7  HPFS/NTFS/exFAT

 

2. device에 mount할 폴더 생성

root@slave1:~# mkdir /app
root@slave1:~# mkdir /app/usb
root@slave1:~# mkdir /app/usb2
root@slave1:~# mkdir /app/hdd

 

3. hdd는 인식가능하게 파티션을 생성해야함..

root@slave1:/# fdisk /dev/sda

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0xb092454e.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


The device presents a logical sector size that is smaller than

the physical sector size. Aligning to a physical sector (or optimal

I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): o

Building a new DOS disklabel with disk identifier 0xb5607451.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


The device presents a logical sector size that is smaller than

the physical sector size. Aligning to a physical sector (or optimal

I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): n

Partition type:

   p   primary (0 primary, 0 extended, 4 free)

   e   extended

Select (default p): p

Partition number (1-4, default 1): 

Using default value 1

First sector (2048-1953525167, default 2048): 

Using default value 2048

Last sector, +sectors or +size{K,M,G} (2048-1953525167, default 1953525167): 

Using default value 1953525167


Command (m for help): w

The partition table has been altered!


Calling ioctl() to re-read partition table.

Syncing disks.


4. 작성된 파티션을 사용하기 위해 파일시스템을 생성한다.

(아래는 전체를 파티션 1개로 설정한 경우임)

root@slave1:/# mkfs -t ext4 /dev/sda1

mke2fs 1.42.5 (29-Jul-2012)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

61054976 inodes, 244190390 blocks

12209519 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=0

7453 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks: 

        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 

        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 

        102400000, 214990848


Allocating group tables: done                            

Writing inode tables: done                            

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done     


*파티션 정보 확인

root@slave1:/# fdisk -l


Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0xb5607451


   Device Boot      Start         End      Blocks   Id  System

/dev/sda1            2048  1953525167   976761560   83  Linux


Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes

4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xf7e9dfe5


        Device Boot      Start         End      Blocks   Id  System

/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux

/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux

root@slave1:/# fdisk -l


Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0xb5607451


   Device Boot      Start         End      Blocks   Id  System

/dev/sda1            2048  1953525167   976761560   83  Linux


Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes

4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xf7e9dfe5


        Device Boot      Start         End      Blocks   Id  System

/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux

/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux


5. device와 폴더 연결(mount)

root@slave1:/# mount /dev/sda1 /app/hdd

 

6. 자동 mount설정

 vi /etc/fstab에 아래의 내용을 추가한다.

 

 /dev/sda1       /app/hdd        ext4    defaults        0       0


번호 제목 날짜 조회 수
110 python 2.6.6에서 print 'A=' 형태의 사용이 python 3.5.1에서 오류(SyntaxError: Missing parentheses in call to 'print') 발생함.. 2016.05.27 2931
109 elasticsearch에서 모든 인덱스(색인)을 삭제하는 방법 2019.06.09 2923
108 초기 오류(java.lang.NoSuchMethodError)에 따른 후속 작업에서 오류(java.lang.NoClassDefFoundError)가 발생되는 상황(quartz에서 주기적으로 작업시) 2016.08.29 2923
107 elasticsearch 1.3.0에서 rdf및 hadoop plugin설치 2016.04.06 2919
106 서버 5대에 solr 5.5.0 설치하고 index data를 HDFS에 저장/search하도록 설치/설정하는 방법(SolrCloud) 2016.04.08 2911
105 S2RDF를 이용한 다른 버젼의 github링크 2016.12.02 2882
104 anaconda3(v5.4)를 이용하여 tensorflow설치후 ipython프로그램을 실행하여 import할때 오류발생시 조치 2018.07.27 2881
103 journalnode노드 기동시 "should be an absolute path"가 발생하고 기동되지 않을 경우 확인사항 2016.09.22 2878
102 hbase startrow와 endrow를 지정하여 검색하기 샘플 2016.12.07 2876
101 Could not configure server becase SASL configuration did not allow the Zookeeper server to authenticate itself properly: javax.security.auth.login.LoginException: Checksum failed 2019.05.18 2870
100 S2RDF 테스트(벤치마크 테스트를 기준으로 python, scala소스가 만들어져서 기능은 파악되지 못함) [1] file 2016.05.27 2865
99 [SparkR]SparkR 설치 사용기 1 - Installation Guide On Yarn Cluster & Mesos Cluster & Stand Alone Cluster file 2016.11.04 2863
98 [Hue metadata]Oracle에 있는 Hue 메타정보 테이블을 이용하여 coordinator와 workflow관계 목록을 추출하는 방법 2023.08.22 2860
97 다중 모듈 프로젝트 설정에 대한 설명 2016.09.21 2851
96 namenode오류 복구시 사용하는 명령 2016.04.01 2851
95 How-to: Build a Complex Event Processing App on Apache Spark and Drools file 2016.10.31 2849
94 servlet-api를 jar형태로 build할때 포함하지 말고 java 설치 위치의 jre/lib/ext에 복사하여 사용하는것이 좋다. 2016.08.10 2840
93 failed to read local state, exiting...오류발생시 조치사항 2016.04.06 2833
92 git설명 한글판 2015.12.09 2829
91 uEnv.txt위치및 내용 2014.07.09 2829
위로