Todd Forsberg's picture
Todd Forsberg - Sat, 2011/08/13 - 20:02

Has anyone found a good solution for file compression for Linux at the filesystem level?  In other words similar to the drive/folder compression of NTFS.

I have one TKL-FileServer which contains mostly text files (or other files which compress very well) and they don't get changed very often.

It would be nice to be able to compress this to save some disk space.

Any ideas?

-Todd

Forum: 
General
Jeremy Davis's picture

Jeremy Davis - Sat, 2011/08/13 - 21:40

According to Wikipedia Btrfs seems to be the only native Linux FS that supports such a feature. I can't 100% confirm but my brief research online suggests that Btrfs support in Ubuntu 10.04 (the basis of TKL v11.x) is only experimental and you are not able to use it as the default root filesystem so without other modification, you would need to maintain ext4(/3/2) for / (root) but could have a Btrfs formatted partition mounted for storage of your shared files.

Googling about to assist you with your question, I came across FuseCompress which seems to acheive the same ends but coming from a different angle which seems to be FS agnostic and doesn't require kernel support.

I can't vouch for either of these solutions and TBH considering how cheap HDDs are, personally I'd probably just buy an additional (or bigger replacement) HDD rather than muck around with this sort of thing, but each to their own. :)

If you do end up trying out either of these (or some other solution you come across) I'd be interested to hear how it all goes for you.

Also, obviously I don't need to stress the importance of good and regular backups when doing this sort of testing, but I'd rather bore you with a reminder than let you forget to do it!

Todd Forsberg's picture

Todd Forsberg - Tue, 2011/08/16 - 02:24

Found a couple more...

http://zfs-fuse.net/

http://e3compr.sourceforge.net/

"E3compr" did not impress me too much, mostly since it seems the project is no longer active.

I was reading a bit about ZFS.  It looks "FULL-Featured" and very nice, but when I got to the list of supported platforms, it stated some sort of licensing issue with the linux kernel.  

Then after searching a bit more I found "ZFS-Fuse".  I'm guessing that it is a port of the ZFS into FuseCompress project.  I'm gonna try it out on a "purely" test VM.

-Todd

-Todd Forsberg

Todd Forsberg's picture

Todd Forsberg - Tue, 2011/08/16 - 07:24

I got it to install ok, but now I'm at a loss as how to configure it.  

I know this is sorta far of the TKL path, but if it works good, maybe an add-on for the FileServer appliance.

-Todd Forsberg

Jeremy Davis's picture

Jeremy Davis - Sat, 2011/08/20 - 05:18

And yes from my reading ZFS is a killer FS and AFAIK ZFS was in the process of being ported to Linux. But when Oracle bought out Sun that apparently stopped dead in it's tracks and now looks like it's unlikely it will be resurected due to licencing (as you suggest).

Todd Forsberg's picture

Todd Forsberg - Sun, 2011/08/14 - 07:44

Thanks.  I found FuseCompress also after posting the msg.  Ya, I think I'll just add another HDD.  I may try out Btrfs just for fun.  If I do, I'll post back here.

-Todd Forsberg

joviermark's picture

joviermark - Thu, 2018/02/08 - 09:23

In terms of availability, 7zip is widely available across UNIX (Linux/BSD/MacOS) and Windows systems. Therefore a zip file is highly portable. xz and 7zip are known to have a better compression algorithm than gzip, but use more memory and time to compress/decompress. when backing up I would recommend tar.7z as it has the highest compression rate saving you space but uses an extra program (7zip). .tar.gz will be larger files and do the same job, you could also use bzip (.tar.bz/bz2) although i'm not sure if that would suit you better as I use gzip or 7zip

Lauryfriese's picture

Lauryfriese - Thu, 2023/09/14 - 06:10
Yes, there are solutions available for file compression at the filesystem level in Linux. One such solution is transparent compression using the LZO or ZFS file systems.LZO File System: The LZO file system is a lightweight, read-only compressed file system. It allows you to create a compressed file system that can be mounted and accessed like any other file system. The compression is transparent, meaning the files are automatically compressed and decompressed on the fly as they are read or written.  

Add new comment