26 January 2011

I've decided to stop using ZFS today...

I have 2 servers where i was using ZFS. One Debian and one FreeBSD. I managed to crash both of them by using ZFS. I have to agree that i was over-using it.

Same hardware for both machines: 1,7 Ghz Intel Pentium 4 CPU with 768 Mb RAM memory and 3x1TB Samsung disks. The test that i was performing was: download one torrent with high amount of peers (multiple disk writes in the same time), with high speed (i have up to 10 mbytes/s - that's 100 mbps connection), in the same time copy some movies over SMB or FTP and start a scrub while doing all of the above...

Debian crashed with kernel panic and didn't reboot itself, i had to manually push the button.
FreeBSD crashed with
panic: kmem_malloc(65536): kmem_map too small: 240738304 total allocated
cpuid = 0
Uptime: 5h18m4s
Cannot dump. Device not defined or unavailable.
Automatic reboot in 15 seconds - press a key on the console to abort
... and then rebooted itself.

I have to say that i tuned the FreeBSD ZFS because it was integrated in the kernel and i was expecting more from it. But i didn't touch the Debian one (fuse is not integrated in kernel). These are the settings that i've done:
vm.kmem_size=1152MB
vfs.zfs.arc_max=768MB
vfs.zfs.prefetch_disable=1
One additional thing that i've noticed is that the load on FreeBSD was 7-8 while writing with 10mbytes/s on the disk (FTP or SMB), while on Debian the load was 0.9-1.1.

Now i'm thinking to chose between:
- mdadm
- LVM

Will come back with test results of both.

No comments:

Post a Comment