메뉴 건너뛰기

Cloudera, BigData, Semantic IoT, Hadoop, NoSQL

Cloudera CDH/CDP 및 Hadoop EcoSystem, Semantic IoT등의 개발/운영 기술을 정리합니다. gooper@gooper.com로 문의 주세요.


1. 현재 연결된 기기 확인

(아래는 31.7GB(/dev/mmcblk0)과 1000.2 GB(/dev/sda)의 2개의 외장기기 확인됨

   /dev/mmcblk0는 /dev/mmcblk0p1과 /dev/mmcblk0p2의 2개 파티션으로 구성 되어 있고

          /dev/sda는 /dev/sda1로 파티션 1개로 구성되어 있음)

root@slave1:~# fdisk -l

Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes
4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xf7e9dfe5

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux
/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux

Disk /dev/sda: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xc197d009

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048  1953521663   976759808    7  HPFS/NTFS/exFAT

 

2. device에 mount할 폴더 생성

root@slave1:~# mkdir /app
root@slave1:~# mkdir /app/usb
root@slave1:~# mkdir /app/usb2
root@slave1:~# mkdir /app/hdd

 

3. hdd는 인식가능하게 파티션을 생성해야함..

root@slave1:/# fdisk /dev/sda

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with disk identifier 0xb092454e.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


The device presents a logical sector size that is smaller than

the physical sector size. Aligning to a physical sector (or optimal

I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): o

Building a new DOS disklabel with disk identifier 0xb5607451.

Changes will remain in memory only, until you decide to write them.

After that, of course, the previous content won't be recoverable.


Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)


The device presents a logical sector size that is smaller than

the physical sector size. Aligning to a physical sector (or optimal

I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): n

Partition type:

   p   primary (0 primary, 0 extended, 4 free)

   e   extended

Select (default p): p

Partition number (1-4, default 1): 

Using default value 1

First sector (2048-1953525167, default 2048): 

Using default value 2048

Last sector, +sectors or +size{K,M,G} (2048-1953525167, default 1953525167): 

Using default value 1953525167


Command (m for help): w

The partition table has been altered!


Calling ioctl() to re-read partition table.

Syncing disks.


4. 작성된 파티션을 사용하기 위해 파일시스템을 생성한다.

(아래는 전체를 파티션 1개로 설정한 경우임)

root@slave1:/# mkfs -t ext4 /dev/sda1

mke2fs 1.42.5 (29-Jul-2012)

Filesystem label=

OS type: Linux

Block size=4096 (log=2)

Fragment size=4096 (log=2)

Stride=0 blocks, Stripe width=0 blocks

61054976 inodes, 244190390 blocks

12209519 blocks (5.00%) reserved for the super user

First data block=0

Maximum filesystem blocks=0

7453 block groups

32768 blocks per group, 32768 fragments per group

8192 inodes per group

Superblock backups stored on blocks: 

        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 

        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 

        102400000, 214990848


Allocating group tables: done                            

Writing inode tables: done                            

Creating journal (32768 blocks): done

Writing superblocks and filesystem accounting information: done     


*파티션 정보 확인

root@slave1:/# fdisk -l


Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0xb5607451


   Device Boot      Start         End      Blocks   Id  System

/dev/sda1            2048  1953525167   976761560   83  Linux


Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes

4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xf7e9dfe5


        Device Boot      Start         End      Blocks   Id  System

/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux

/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux

root@slave1:/# fdisk -l


Disk /dev/sda: 1000.2 GB, 1000204886016 bytes

81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Disk identifier: 0xb5607451


   Device Boot      Start         End      Blocks   Id  System

/dev/sda1            2048  1953525167   976761560   83  Linux


Disk /dev/mmcblk0: 31.7 GB, 31657558016 bytes

4 heads, 16 sectors/track, 966112 cylinders, total 61831168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0xf7e9dfe5


        Device Boot      Start         End      Blocks   Id  System

/dev/mmcblk0p1            2048       40960       19456+  81  Minix / old Linux

/dev/mmcblk0p2           40961     7577599     3768319+  81  Minix / old Linux


5. device와 폴더 연결(mount)

root@slave1:/# mount /dev/sda1 /app/hdd

 

6. 자동 mount설정

 vi /etc/fstab에 아래의 내용을 추가한다.

 

 /dev/sda1       /app/hdd        ext4    defaults        0       0


번호 제목 날짜 조회 수
210 python실행시 ValueError: zero length field name in format오류 해결방법 2016.05.27 3929
209 S2RDF 테스트(벤치마크 테스트를 기준으로 python, scala소스가 만들어져서 기능은 파악되지 못함) [1] file 2016.05.27 2865
208 CentOS6에 python3.5.1 소스코드로 빌드하여 설치하기 2016.05.27 4113
207 RDF storage조합에대한 test결과(4store, Jena+HBase, Hive+HBase, CumulusRDF, Couchbase) 페이지 링크 2016.05.26 3018
206 spark-submit으로 spark application실행하는 다양한 방법 2016.05.25 4414
205 spark 온라인 책자링크 (제목 : mastering-apache-spark) 2016.05.25 4248
204 "Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources"오류 발생시 조치사항 2016.05.25 4563
203 spark-env.sh에서 사용할 수있는 항목. 2016.05.24 4379
202 Spark 1.6.1 설치후 HA구성 2016.05.24 4533
201 spark-shell실행시 "A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection."오류가 발생하는 경우 해결방법 2016.05.20 2953
200 Master rejected startup because clock is out of sync 오류 해결방법 2016.05.03 5074
199 kafka broker기동시 brokerId가 달라서 기동에 실패하는 경우 조치방법 2016.05.02 5189
198 kafka 0.9.0.1 for scala 2.1.1 설치및 테스트 2016.05.02 4052
197 solr 인스턴스 기동후 shard에 서버가 정상적으로 할당되지 않는 경우 해결책 2016.04.29 4036
196 collection생성시 -shards와 -replicationFactor값을 잘못지정하면 write.lock for client xxx.xxx.xxx.xxx already exists오류가 발생한다. 2016.04.28 3211
195 solrdf초기 기동시 "Caused by: java.lang.IllegalAccessError: tried to access field org.apache.solr.handler.RequestHandlerBase.log from class org.gazzax.labs.solrdf.handler.update.RdfUpdateRequestHandler" 오류가 발생시 조치사항 2016.04.22 4134
194 solrcloud에 solrdf1.1설치하고 테스트 하기 2016.04.22 3287
193 Spark 2.1.1 clustering(5대) 설치(YARN기반) 2016.04.22 4960
192 Hadoop 완벽 가이드 정리된 링크 2016.04.19 4354
191 cumulusRDF 1.0.1설치및 "KeyspaceCumulus" keyspace확인하기 file 2016.04.15 8730
위로