Cannot get rid of root filesystem LVM snapshot

General Interworx Control Panel Info General Inquries
1

Hi all,

I am writing here with hope somebody had already run and successfully resolved this issue. I am on CentOS 4.4 with kernel kernel-smp-2.6.9-200.0.2.EL.iworx.x86_64.rpm provided by InterWorx. I’ve been in the process of creating a backup script which uses LVM snapshoting feature to backup a snapshot of root filesystem instead of live filesystem.

During testing phase on testing CentOS installation, I’ve been able to successfully create, mount, unmount and remove root filesystem snapshots. On live production system, I’ve also successfully create, mounted and unmounted root filesystem snapshot, but I CANNOT REMOVE IT in any way.

lvremove -f gives following output:

File descriptor 3 left open
File descriptor 5 left open
File descriptor 7 left open
/dev/cdrom: open failed: Read-only file system
Can’t remove open logical volume "SSsystem

tried also to deactivate the snapshot using lvchange -an command:

File descriptor 3 left open
File descriptor 5 left open
File descriptor 7 left open
/dev/cdrom: open failed: Read-only file system
Can’t change snapshot logical volume “SSsystem”

lvdisplay outputs this:

File descriptor 3 left open
File descriptor 5 left open
File descriptor 7 left open
— Logical volume —
LV Name /dev/VGdb/LVdb
VG Name VGdb
LV UUID R4mtQ4-Idyo-SRiU-EfLg-TFM6-NZN2-Cd1kgv
LV Write Access read/write
LV Status available

open 1

LV Size 14.66 GB
Current LE 469
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:1

— Logical volume —
LV Name /dev/VGsystem/LVsystem
VG Name VGsystem
LV UUID HusBAX-tbxn-6283-zap5-Cihe-K1AX-jKDbk7
LV Write Access read/write
LV snapshot status source of
/dev/VGsystem/SSsystem [active]
LV Status available

open 1

LV Size 31.25 GB
Current LE 1000
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0

— Logical volume —
LV Name /dev/VGsystem/SSsystem
VG Name VGsystem
LV UUID qohvbp-yo1c-SFGv-suM1-9QC9-OwA3-TpX1i0
LV Write Access read/write
LV snapshot status active destination for /dev/VGsystem/LVsystem
LV Status available

open 1

LV Size 31.25 GB
Current LE 1000
COW-table size 2.66 GB
COW-table LE 85
Allocated to snapshot 2.32%
Snapshot chunk size 8.00 KB
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:4

According to various newsgroups it seems LVM2 has some problems with root filesystem snapshots, but most of them refers to freeze during snapshot creating which is not my case, but many of them are also speaking about loosing the filesystem etc.

At this very time I am in a situation when the snapshot space is slowly filling and I am afraid to reboot, manually fill the snapshot (with hope it will automatically deactivate) or anything else, because this is live production system placed in data center and currently I have no physical access to it.

I’ll be grateful for any ideas which might help to resolve this problem. Thanks very much.

Muad

Here is output of uname -a:

Linux xxxx.xxx.xxx 2.6.9-200.0.2.EL.iworxsmp #1 SMP Fri Aug 4 15:25:06 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux

2

Just to let people know that it is not good idea to try to create LVM snapshots of root filesystem, this is the reply from Alasdair on linux-lvm mailing list:

The tools do not support root snapshots properly yet - whether or not they
work is down to chance. To remove one reliably you need lvremove from a
rescue environment - you’re lucky lvremove is refusing to proceed - if it
got further it could hang your machine. In principle you can recover
without rebooting but you’ll need to write a custom script to run various
dmsetup commands from a ramdisk, for example, and then fix up the lvm2
metadata with activation disabled. [If you learn how snapshots are
created/destroyed you can work out what to do: sorry, it’s too much for me
to explain here. This is what the tools will want enhancing to do to
provide proper support.]

Older kernels like yours however can get into a state where recovery is only
possible by writing directly to kernel memory, if you already ran certain
commands in the wrong sequence.

Alasdair

3

I can’t stand LVMs; I avoid them like the plauge :stuck_out_tongue: