Linux Format forums

[jjmac]

Howdy all,


defrag disk defragmentation file-systems
========================================

Something i saw in a thread on an other site, discussing fragmentation issues in Linux.

And, as there seems to be a number of people who visit here and are also involved in site maintenance/administration -=- I thought it would be interesting, to read about their thoughts on this.


I'm hoping we can bypass the basics. Such as different file systems for different purposes and the general good design of Unix like/Linux file systemsin general ie; block groups.

(But if we must then i guess we must ...)

And focus more on the strategies people use to address this.

It would seem to me to only really be of concern to a system where the creation and removal of files involves writes of largely differing block numbers. Making it not always possible to find a continuous block sequence to fit. Along with a fairly good whack of file size increase due to editing activity. In which case a fragmentation situation would be unavoidable. If just left to itself, that is.

Either way it seems to me that blocks have just got to be moved about eventually, probably large numbers, if a system is prone to this type of thing. Which, tends to spell 'tar' to me.

So ...

# What kind of problems have you struck ?

# What solution(s) did you use to fix them --- defrag tools etc ?

# programs ?

# Which ones sux, which ones don't ?

# Did you ever leave it until it just got so bad ... ?





jm
_________________
http://counter.li.org
#313537

The FVWM wm -=- www.fvwm.org -=-

Somebody stole my air guitar, It happened just the other day,
But it's ok, 'cause i've got a spare ...

[Dutch_Master]

I'm having a bit of an issue on understanding what you're after here?

1) Linux/Unix filesystems are far superior to anything from Redmond. So much so that fragmentation is rarely an issue, likewise defragging the system.
2) Based on 1) Linux/Unix users never have to defrag their systems, at least not on purpose.
3) I've never heard of a defrag-tool for anything other then DOS and NTFS based systems, I suppose because no-one needs them...

Mind that without understanding the basics, a sensible discussion could not happen...

[nelz]

More recent filesystems, like ext4 and btrfs, leave empty blocks at the end of a file, so it can grow without fragmentation.
_________________
Unix is user-friendly. It's just very selective about who it's friends are.

[gerdesj]

[Rhakios]

From time to time, file systems get checked at boot up. If you don't have one of those flashy framebuffer boot screens in the way, you can see a result flash up of % discontinuity of files. This usually tends to be pretty low on file systems that have at least 30% free space and aren't seeing too many re/writes, usually less that 10%.
Installing a new OS once a year seems to keep it low, only the home directory is ever around, unformatted, on any of my machines for much longer than that. Having said which, my home folder was only showing about 7% discontinuity last time I saw it checked, so I don't find it a huge problem.
The question is whether or not it's worth it for file systems that get fuller than 70%, or which stick around with lots of file turnover for a long time. As jj suggests, a tar backup, formatting the partition and then untarring the data back would be one way of solving the problem. Shake looks like an interesting little utility with some neat features.
Of course, what we really need is benchmarks.
_________________
Bye, Rhakios

[gerdesj]

A quick re read later of the top post.

I personally rarely bother worrying about fragmentation on my "work" systems. By the time fragmentation might be an issue, we probably would have talked the customer into new hardware! In a corporate environment typically h/w is written off over three years. Rarely does a machine last > 5 years before moving to new hardware.

When I am looking at an old crusty system, then to try and wring a bit more performance out, I'd start by analyzing the workload and killing off old stuff.

To be honest defrag has never featured high on my list of things to do. I have predominately used reiser3 for file systems and I don't think that fragmentation has been a real issue.

I have used Shake a few times but only on my personal systems. I certainly don't do the Vista thing of running it every time a new piece of software is installed 9)

[jjmac]

Howdy all,

And thanks for the feed back. I'd mention the actual site and thread but that tends to sit a bit uneasily with me. Though i suppose it shouldn't.


In any case, it was one of those recently installed Linux and is threatening to go back to windows over a defrag issue he/she was having.

They said their system was involved high volume usage involving large (30M) files.

It did seem somewhat strange to me. Especially as the 'maintainer' appeaed to have just done a default Linux install, and left it at that. And was claiming the file system was running with a 60% discontinuity. Specifics had to be probed out of him as well i might add .

I could see their point, but it did seem to me that regardless of the whys and wheres --- the real problem they had was having a lack of stategy to deal with the situation. For if or when it should arise.


Dutch_Master wrote:
>>
I'm having a bit of an issue on understanding what you're after here?
2) Based on 1) Linux/Unix users never have to defrag their systems, at least not on purpose.
>>

Just some feed back from peoples experiences along these lines. As per the post.

On #1 --- the keyword being 'rarely' which is what this here is about.
On #2 --- to broad a statement and the point that the other thread poster was using as a 'it's not my fault' excuse . 'Never' is really only a general purpose statement, and so can be a bit mis-leading.
On #3 --- Either have i. Except for Con Kolivas' defrag script. But it isn't the kind of thing a person couldn't put a script together themselves for, and then, of course add elaborations to over time. Such as cancelling inode flags, using a dedicated write-to and write-from partition etc.


Dutch_Master wrote:
>>
Mind that without understanding the basics, a sensible discussion could not happen...
>>

And get bogged down in too ... i was hoping to assume responces were already inclusive in that regard . Which, not surprisingly, they are.



nelz wrote:
>>
More recent filesystems, like ext4 and btrfs, leave empty blocks at the end of a file, so it can grow without fragmentation.
>>

Yes, ext4 was mentioned. But not the added facility.
btrfs wasn't mentioned, and is a new one to me.



Rhakios wrote:
>>
From time to time, file systems get checked at boot up. If you don't have one of those flashy framebuffer boot screens in the way, you can see a result flash up of % discontinuity of files. This usually tends to be pretty low on file systems that have at least 30% free space and aren't seeing too many re/writes, usually less that 10%.
>>

I like watching it all scroll by too . And the free space that the person had available was one thing that hadn't been mentioned ...


Just on the % discontinuous return that fsck comes up with. It just goes to show, that it is up to the person to be able to interpret what the tool is trying to tell them. That is, they need to be in control of their system, rather than the other way round.

Such as --- for say, a 4G partition, with a standard new ext2 file-system created on it with a 1% reserve ... the discontinuous percentage should be around 9.x%. Falling with further usage. That's purely due to the dispersal of meta-data writes at the beginning of the array of 'block-groups' which make up the file-system. And doesn't lend itself to any defrag issue at all.

I'll have to go and mention that on the other thread :evil' (grin).

Thanks again for all your feed back -=- of course -=-




jm
_________________
http://counter.li.org
#313537

The FVWM wm -=- www.fvwm.org -=-

Somebody stole my air guitar, It happened just the other day,
But it's ok, 'cause i've got a spare ...