메뉴 건너뛰기

Bigdata, Semantic IoT, Hadoop, NoSQL

Bigdata, Hadoop ecosystem, Semantic IoT등의 프로젝트를 진행중에 습득한 내용을 정리하는 곳입니다.
필요한 분을 위해서 공개하고 있습니다. 문의사항은 gooper@gooper.com로 메일을 보내주세요.


#!/bin/bash

kinit -k -t /var/lib/keytab/hadoop.keytab hadoop@GOOPER.COM


usage="Usage: del_diff.sh [days] [delete target path]"


if [[ $# != 2 ]]

then

   echo $usage

   exit 1

fi


path=${2}

echo "delete target path : " $path


now=$(date +%s)

echo "old file date point : " $now


hdfs dfs -ls $path | while read f; do

dir_date=`echo $f | awk '{print $6}'`

#180일

difference=$(( ( $now - $(date -d "$dir_date"  +%s) ) / (24 * 60 * 60) ))

if [ $difference -gt $1 ]; then

  if [[ `echo $f | awk '[ print $8 }'` == *$path* ]]; then

      hdfs dfs -rm `echo $f| awk '{ print $8 }'`;

  else

     echo 'else - target path is [' `echo $f| awk '{ print $8 }'` '] not contain this path [' $path ']'

  fi

fi

done

A personal place to organize information learned during the development of such Hadoop, Hive, Hbase, Semantic IoT, etc.
We are open to the required minutes. Please send inquiries to gooper@gooper.com.

위로