Posted in Linux

用 restic 備份 (snapshot) 資料

用 restic 備份 (snapshot) 資料 Posted on 2019 年 03 月 25 日Leave a comment

restic 是一個 GO 寫的 snapshot 備份工具,支援 Linux 、 Windows 平台,下指令方式備份資料,也支援加密資料

安裝 on CentOS7

yum-config-manager --add-repo https://copr.fedorainfracloud.org/coprs/copart/restic/repo/epel-7/copart-restic-epel-7.repo
yum install -y restic

版本

restic version

自我更新

restic self-update

或 source 安裝 (記得安裝 Golang )

git clone https://github.com/restic/restic
cd restic
go run -mod=vendor build.go

一開始,我們要建一組密碼,與 init 一個備份資料庫 (備份目地為 /backup/restic,還有引用 password 檔)

mkdir /etc/restic
echo 'mypassword' > /etc/restic/.password
restic --repo /backup/restic --password-file=/etc/restic/.password init

成功產生的訊息

created restic repository 04dc13cc7f at /backup/restic

假如不下 –password-file ,restic 也支援使用變數

RESTIC_PASSWORD=123456
或
RESTIC_PASSWORD_FILE=/etc/restic/.password

restic 使用這組密碼來加密備份資料,所以要好好保存密碼,不然密碼不見等於資料不見

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

開始備份 backup 來源

restic --repo /backup/restic --password-file=/etc/restic/.password backup /etc/passwd
open repository
repository 04dc13cc opened successfully, password is correct
created new cache in /root/.cache/restic

Files:           1 new,     0 changed,     0 unmodified
Dirs:            1 new,     0 changed,     0 unmodified
Added to the repo: 3.009 KiB

processed 1 files, 2.188 KiB in 0:00
snapshot 4f1bec25 saved

也可以指定要備份的清單,restic 去讀這個檔案

find /etc | grep passwd > /etc/resitc/include_file.txt
restic --repo /backup/restic --password-file=/etc/restic/.password backup --files-from /etc/resitc/include_file.txt

備份 mysql

mysqldump --opt -u root mysql | restic --repo /backup/restic --password-file=/etc/restic/.password backup --stdin 

# 或

mysqldump --opt -u root mysql | restic --repo /backup/restic --password-file=/etc/restic/.password backup --stdin --stdin-filename mysql.sql

後者多了 –stdin-filename 有啥不同

如果用 shapshots 查看備份清單,會看到它的 Paths 上面的是 /root/stdin 下面的是 /root/mysql.sql,且還原時也是用它的名稱,所以建議用上 –stdin-filename

查看備份清單

restic --repo /backup/restic --password-file=/etc/restic/.password snapshots
repository 04dc13cc opened successfully, password is correct
ID        Time                 Host          Tags        Paths
------------------------------------------------------------------------
4f1bec25  2019-03-23 07:19:44  localhost              /etc/passwd
a1680bb5  2019-03-23 07:22:09  localhost              /var/lib/docker
aa5ea708  2019-03-23 08:06:43  localhost              /root/mysql.sql
392b564c  2019-03-23 08:18:20  localhost              /root/stdin
------------------------------------------------------------------------

還原

restic --repo /backup/restic --password-file=/etc/restic/.password restore 4f1bec25 --target /root/restore

如果還原 /etc/passwd 這個,匯出來的路徑會長這樣子 /root/restore/etc/passwod

所以如果確定要還原的話,可以直接指向 –target /

如果不指定 ID,給 latest ,它會直接還原最新時間的那一筆,

restic --repo /backup/restic --password-file=/etc/restic/.password restore latest --target /root/restore

或者再加上 –path /etc/passwd,就是還原那個 Paths 的最新一筆

restic --repo /backup/restic --password-file=/etc/restic/.password restore latest --path /etc/passwd --target /root/restore

restic 另提供掛載的功能,不需要實際還原檔案才能看到內容,可以用掛載方式查看內容再還原也行,

restic --repo /backup/restic --password-file=/etc/restic/.password mount /root/restore/mount

如果出現錯誤,是因為系統需要 fuse 套件

repository 04dc13cc opened successfully, password is correct
fusermount: exec: "fusermount": executable file not found in $PATH
unable to umount (maybe already umounted?): exec: "fusermount": executable file not found in $PATH

裝一下 fuse

yum install -y fuse

這樣子就可以成功載掛,但它是前景執行的,Ctrl+C 中斷後,掛載就沒了

repository 04dc13cc opened successfully, password is correct
Now serving the repository at /root/restore/mount
Don't forget to umount after quitting!

可以看到 restic mount 起來的樣子,就可以根據 host, id, snapshot, tag 去找備份內容

dr-xr-xr-x 1 root root 0  3月 25 10:48 hosts
dr-xr-xr-x 1 root root 0  3月 25 10:48 ids
dr-xr-xr-x 1 root root 0  3月 25 10:48 snapshots
dr-xr-xr-x 1 root root 0  3月 25 10:48 tags

備份刪除、週期

restic 的每一個時間點的備份就是完整備份,所以來源檔案的新增刪除修改都會映照在備份裡面。不過有時後舊的時間點裡面的檔案來源早已不需要了,我們也就不用再保留了
restic 提供刪除的功能 (根據 ID)

restic --repo /backup/restic --password-file=/etc/restic/.password forget bdbd3439

顯示

repository 04dc13cc opened successfully, password is correct
removed snapshot 47583656

然而 forget 只是刪除快照而已,真正資料並沒有刪除
所以還要下 prune

restic --repo /backup/restic --password-file=/etc/restic/.password prune
repository 04dc13cc opened successfully, password is correct
counting files in repo
building new index for repo
[0:00] 100.00% 6 / 6 packs
repository contains 6 packs (7 blobs) with 1010.057 KiB
processed 7 blobs: 0 duplicate blobs, 0 B duplicate
load all snapshots
find data that is still in use for 0 snapshots
[0:00] 0 / 0 snapshots
found 0 of 7 data blobs still in use, removing 7 blobs
will remove 0 invalid files
will delete 6 packs and rewrite 0 packs, this frees 1009.593 KiB
counting files in repo
[0:00] 0 / 0 packs
finding old index files
saved new indexes as []
remove 1 old index files
[0:00] 100.00% 6 / 6 packs deleted
done

不然就下 forget 再加上 –prune

restic --repo /backup/restic --password-file=/etc/restic/.password forget bdbd3439 --prune

模擬刪除測試

一開始先備份一次

restic --repo /backup/restic --password-file=/etc/restic/.password backup /var/www/html

新增一個檔案

cd /var/www/html ; touch `date +%F.%T`

再備份一次

restic --repo /backup/restic --password-file=/etc/restic/.password backup /var/www/html

刪除最舊的一筆

restic --repo /backup/restic --password-file=/etc/restic/.password forget d9510ae7 --prune

查看內容
結果我們新增的檔案有在,其它的也在

restic --repo /backup/restic --password-file=/etc/restic/.password snapshots
restic --repo /backup/restic --password-file=/etc/restic/.password mount /root/restore/mount
ll /root/restore/mount/ids/XXXX/var/www/html/

除了用 ID 一個一個刪之外,restic 也有週期的功能, 用 –keep-last 1 刪除備份並只保留最新一份

restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-last 1

其它保留的參數

  -l, --keep-last n            keep the last n snapshots
  -H, --keep-hourly n          keep the last n hourly snapshots
  -d, --keep-daily n           keep the last n daily snapshots
  -w, --keep-weekly n          keep the last n weekly snapshots
  -m, --keep-monthly n         keep the last n monthly snapshots
  -y, --keep-yearly n          keep the last n yearly snapshots
      --keep-within duration   keep snapshots that are newer than duration (eg. 1y5m7d2h) relative to the latest snapshot
      --keep-tag taglist       keep snapshots with this taglist (can be specified multiple times) (default [])

模擬備份週期

一般情況下,你要一直保留備份資料也行,但看備份空間夠不夠,如果夠,就一直保留下去吧,
基本上 restic 是 snapshot 備份,同一資料並不會累加備份的。

這邊要模擬只保留幾份的情形,

先產生出 30 天份的備份資料

ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
e441f024  2019-02-24 00:15:00  localhost              /var/www/html
77b099ab  2019-02-25 00:15:00  localhost              /var/www/html
df0de8de  2019-02-26 00:15:00  localhost              /var/www/html
a6ccac31  2019-02-27 00:15:00  localhost              /var/www/html
a04061c2  2019-02-28 00:15:00  localhost              /var/www/html
01802265  2019-03-01 00:15:00  localhost              /var/www/html
76353657  2019-03-02 00:15:00  localhost              /var/www/html
6ee27a29  2019-03-03 00:15:00  localhost              /var/www/html
29ea0839  2019-03-04 00:15:00  localhost              /var/www/html
42a768f2  2019-03-05 00:15:00  localhost              /var/www/html
ee4ba2ca  2019-03-06 00:15:00  localhost              /var/www/html
321bb6ec  2019-03-07 00:15:00  localhost              /var/www/html
d2d5a108  2019-03-08 00:15:00  localhost              /var/www/html
89eaeaa8  2019-03-09 00:15:00  localhost              /var/www/html
752c23ef  2019-03-10 00:15:00  localhost              /var/www/html
b9fadfb0  2019-03-11 00:15:00  localhost              /var/www/html
648b2187  2019-03-12 00:15:00  localhost              /var/www/html
5ce64a63  2019-03-13 00:15:00  localhost              /var/www/html
38fca5c3  2019-03-14 00:15:00  localhost              /var/www/html
22975717  2019-03-15 00:15:00  localhost              /var/www/html
101aa3f3  2019-03-16 00:15:00  localhost              /var/www/html
fee4f2fc  2019-03-17 00:15:00  localhost              /var/www/html
96384718  2019-03-18 00:15:00  localhost              /var/www/html
bf7c0d2b  2019-03-19 00:15:00  localhost              /var/www/html
2ffc03de  2019-03-20 00:15:00  localhost              /var/www/html
cb9871d5  2019-03-21 00:15:00  localhost              /var/www/html
d4e6f336  2019-03-22 00:15:00  localhost              /var/www/html
ef9bb129  2019-03-23 00:15:00  localhost              /var/www/html
7fe99e44  2019-03-24 00:15:00  localhost              /var/www/html
c0b6f991  2019-03-25 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
30 snapshots

保留 7 天

restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-daily 7

結果

ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
bf7c0d2b  2019-03-19 00:15:00  localhost              /var/www/html
2ffc03de  2019-03-20 00:15:00  localhost              /var/www/html
cb9871d5  2019-03-21 00:15:00  localhost              /var/www/html
d4e6f336  2019-03-22 00:15:00  localhost              /var/www/html
ef9bb129  2019-03-23 00:15:00  localhost              /var/www/html
7fe99e44  2019-03-24 00:15:00  localhost              /var/www/html
c0b6f991  2019-03-25 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
7 snapshots

如果是 weekly 呢

(它是以星期日為主)

restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-weekly 4
ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
cdaa1364  2019-03-10 00:15:00  localhost              /var/www/html
b781c508  2019-03-17 00:15:00  localhost              /var/www/html
3d12fa8f  2019-03-24 00:15:00  localhost              /var/www/html
dcefdd63  2019-03-25 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
4 snapshots

再來是月 (這邊我產出 120 天的資料)

(月則是拿月的最後一天為主)

restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-monthly 3
ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
76970400  2019-01-31 00:15:00  localhost              /var/www/html
5b9b8ac0  2019-02-28 00:15:00  localhost              /var/www/html
4461bc3e  2019-03-25 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
3 snapshots
那如果我日周月全滲在一起呢,用 --keep-within 參數

但不是我想的如此,它不是天只保留 7 天份,周只保留 4 周份、月只保留 3 月份
restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-within 3m4w7d

120 天的備份檔只移掉如下,往後的時間都留著

remove 23 snapshots:
ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
47e8df45  2018-11-25 00:15:00  localhost              /var/www/html
c42afaa4  2018-11-26 00:15:00  localhost              /var/www/html
3d5cba91  2018-11-27 00:15:00  localhost              /var/www/html
67cb77e0  2018-11-28 00:15:00  localhost              /var/www/html
0c57d30f  2018-11-29 00:15:00  localhost              /var/www/html
d3d534ae  2018-11-30 00:15:00  localhost              /var/www/html
95371630  2018-12-01 00:15:00  localhost              /var/www/html
2c399fa5  2018-12-02 00:15:00  localhost              /var/www/html
c36a1b3d  2018-12-03 00:15:00  localhost              /var/www/html
1638bd72  2018-12-04 00:15:00  localhost              /var/www/html
8bba7132  2018-12-05 00:15:00  localhost              /var/www/html
854a98c9  2018-12-06 00:15:00  localhost              /var/www/html
26edee90  2018-12-07 00:15:00  localhost              /var/www/html
89b3a176  2018-12-08 00:15:00  localhost              /var/www/html
91cde722  2018-12-09 00:15:00  localhost              /var/www/html
cc19d2fa  2018-12-10 00:15:00  localhost              /var/www/html
fc87e75c  2018-12-11 00:15:00  localhost              /var/www/html
2e6a1035  2018-12-12 00:15:00  localhost              /var/www/html
242ba1a1  2018-12-13 00:15:00  localhost              /var/www/html
10f884d5  2018-12-14 00:15:00  localhost              /var/www/html
c02cfbaa  2018-12-15 00:15:00  localhost              /var/www/html
0938ce5d  2018-12-16 00:15:00  localhost              /var/www/html
4cd7a66d  2018-12-17 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
23 snapshots

不過不過,用這個 (就理想多了)

restic --repo /backup/restic --password-file=/etc/restic/.password forget --prune --keep-daily 7 --keep-weekly 4 --keep-monthly 3
ID        Time                 Host          Tags        Paths
----------------------------------------------------------------------
4bbf62da  2019-01-31 00:15:00  localhost              /var/www/html
b5612d39  2019-02-28 00:15:00  localhost              /var/www/html
51b1315f  2019-03-10 00:15:00  localhost              /var/www/html
071c51ed  2019-03-17 00:15:00  localhost              /var/www/html
7d953422  2019-03-19 00:15:00  localhost              /var/www/html
14459f59  2019-03-20 00:15:00  localhost              /var/www/html
5e8b1d62  2019-03-21 00:15:00  localhost              /var/www/html
6e6fd04e  2019-03-22 00:15:00  localhost              /var/www/html
59dc4daf  2019-03-23 00:15:00  localhost              /var/www/html
a2987ae6  2019-03-24 00:15:00  localhost              /var/www/html
d4590e6a  2019-03-25 00:15:00  localhost              /var/www/html
----------------------------------------------------------------------
11 snapshots

發佈留言

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *