remove thousands of files from a directory

[sde]

i have a directory with about 650,000 files in it that i need to remove. is the best way to do it with find piped to xargs?

[sde]

ls | xargs rm

this worked much faster than find

[ShadyCraig]

what's wrong with

rm *

[sde]

Originally Posted by ShadyCraig

what's wrong with

rm *

that's what i tried first. it can't handle that many files.

[redhead]

# find . -exec rm -f {} \;

Would be my choice.

[Belisarius]

Can't you just go up a directory then "rm -rf <dir>"?

Barring that, redhead's idea would be my next choice.

[sde]

Originally Posted by Belisarius

Can't you just go up a directory then "rm -rf <dir>"?

Barring that, redhead's idea would be my next choice.

no, that's why i needed an alternative. 650,000 files is too many for rm to handle.

find was taking forever. ls | xargs completed within a minute.

[ShadyCraig]

Originally Posted by sde

that's what i tried first. it can't handle that many files.

I though it might be, but I didn't want to try it to find out

Cheers

[sde]

so, how does one get 650,000 files in the first place? i had a user on my server who never checked their system account for the last year or so. their mail file got very big.

i recently installed dovecot, which puts each email into an individual file, instead of all emails in 1 file, and vua la. 650k message is now 650k files

[DJMaze]

650k file != 650k files * 4k

Sometimes it's better to have 1 file instead of 650k because a fle of 1kb takes up 4kb of space.

Based on file system this can vary from 1k (Raiser) to 64k (FAT32).

That's why you always format a drive based on what tasks a server may do

[sde]

no, that was supposed to be 650k messages, i.e. emails in that spool file. it was actually a couple GB in size.

[teknomage1]

Find - exec is much slower than find |xargs rm, because in the case of n files find -exec will invoke rm n times whereas find|xargs will only invoke rm n mod <max commandline args> times, so you see a major speed up in terms of process invocation overhead.

The ultimate power combo is find -print0|xargs -0 rm -f
It uses null characters rather than whitespace to separate the output/input so it won't break in the face of files with spaces or newlines embedded in the names.