SliTaz Forum

shann

Administrator

Registered: 2011-04-01

Posts: 1,296

Website

[ISSUE] Pangolin instance

Hi,

Pangolin occured issue today around 1:00 PM, 503 service unavailable for hg.slitaz.org / forum.slitaz.org.

Suspect issue on private service provider between DC, but other traffic work for mirror and tank.

Connect on kvm to first node who host pangolin, and discover boot stuck on grub.

Reboot it to live cdrom, can't mount filesystem.

Need investigate what happen, infact hypervisor system are good, filesystem also, on same hypervisor we have mirror that no issue occured.

I restore backup daily doing this day midnight on second hypervisor.

second partition seem no impacted, i restore mysql database from datas before crash.

Sorry for apology

Stanislas (shann)

shann

Administrator

Registered: 2011-04-01

Posts: 1,296

Website

Re: [ISSUE] Pangolin instance

Hi,

After more investigations, think found cause but need to plan intervention :

mirror zfs state HEALTHY

SMART test on disks PASSED fort both disk

But for second disk see have 580 reallocated_sector, 0 current_pending_sector, 0 udma_crc.

In case i nothing receive because all return good state

At time pangolin run on other node, remain mirror vm on it, check to move on another node to avoid break this.

shann

Administrator

Registered: 2011-04-01

Posts: 1,296

Website

Re: [ISSUE] Pangolin instance

Hi,

Disk replaced, check it's ok for SMART and sector reallocated value.

- ZFS resilvered for system / vm_data mirror partitions

- EFI partitions synced

We are come back to safe state.

VMs come back to original node.

shann

Administrator

Registered: 2011-04-01

Posts: 1,296

Website

Re: [ISSUE] Pangolin instance

Hi,

Issue this night after backup of pangolin VM.

VM stuck on grub boot 'error 17', after investigation, first partition seem corrupted.

impossible to mount sda1, partition see as configfs, but sda2 that store /home, mount successfully (see clean filesystem).

Check dump himself it's ok, i restore it.

For dump, we use snapshot method, except this time don't have issue with it, read on proxmox forum could be happen when qemu agent set but not running, for SliTaz VM we don't have it.

Maybe check to use stopped method (implied downtime) at least for pangolin vm that store forum and database.

Sorry for issue