How to rationalize duplicates across 11 drives

If you can get all your files onto a single logical drive, a tool like
FreeDups should be able to take care of the duplicate files.

<http://www.stearns.org/freedups/>

If you need something more progressive in how it applies its de-duplication,
you could just as easily start with two consolidated drives, run freedups,
then add another drive, run it again, wash/rinse/repeat until all 11 drives
are consolidated.

Naturally, differences in dates and other criteria might cause it to miss
some dupes, but in my own usage, it's been quite a solid utility. It also
has the advantage of leaving all the instances of your files intact, but it
replaces multiple instances with hard links, so they only exist once on disk
but can appear multiple times in your directory structure.

--
Nik :: gerberinik.net
Software picks, serious Mac geekery and productivity tips!
<http://iNik.net/>

On 2007-03-20, at 09:46, ldh wrote:

> There are thousands of duplicate photo files in multiple drives of
> photos.

Some people! ;-)

GraphicConverter is worth checking out. From the manual:

> 3.3.3.2 Browser Find duplicate files
> This functions searches for duplicate files starting from the
> current viewed folder in the
> browser.
> You get a progress dialog first:
>
> Now you get a dialog that displays all duplicates. [with previews]

It works on both identical files and similar files (with a tolerance
setting). Deletion can be manual or automatic based on a specified
criteria.

On 19-Mar-2007, at 15:46, ldh wrote:
> Any suggestions?

MD5 hash of every file?

MD5 is not guaranteed to be unique, but the chances of finding a
duplicate MD5 hash on identically sized files is vanishingly small.

On 21-Mar-2007, at 13:06, Tom Robinson wrote:
> On 2007-03-20, at 09:46, ldh wrote:
>> There are thousands of duplicate photo files in multiple drives of
>> photos.
>
> Some people! ;-)
>
> GraphicConverter is worth checking out. From the manual:
>
>> 3.3.3.2 Browser Find duplicate files
>> This functions searches for duplicate files starting from the
>> current viewed folder in the browser.

And there's the rub, no? That doesn't allow you to find duplicates
across multiple drives, but just in a single directory tree.

Or am I missing something?

On 2007-03-22, at 10:33, Google Kreme wrote:

>> GraphicConverter is worth checking out. From the manual:
>>
>>> 3.3.3.2 Browser Find duplicate files
>>> This functions searches for duplicate files starting from the
>>> current viewed folder in the browser.
>
> And there's the rub, no? That doesn't allow you to find duplicates
> across multiple drives, but just in a single directory tree.
>
> Or am I missing something?

You missed my unspoken assumption there would be a manual high-level
combine of the drives first ;-)

Somebody on the GraphicConverter mailing list might have a better
idea, or Thorsten (the author) might even be taken enough to code it
(as he sometimes is). <gcmacyahoogroups.com>

Cheers

On Mar 22, 2007, at 2:21 PM, Tom Robinson wrote:

> You missed my unspoken assumption there would be a manual high-level
> combine of the drives first ;-)

I, too, missed your unspoken assumption about a manual high-level
combine of drives. Your first post was the final (but not the only)
straw causing me to purchase Graphic Converter. I'm pleased with its
ability to find duplicates, but my main reason for purchase was to
find duplicates on multiple drives.
HOW CAN a senior citizen (born before 1935) learn how to make a
"manual combine of multiple drives"?

On 23-Mar-2007, at 11:43, Power Donald wrote:
> HOW CAN a senior citizen (born before 1935) learn how to make a
> "manual combine of multiple drives"?

Will GC look at a "smart folder"?

If not that, the try this, create a new folder on your desktop, or
somewhere, Command-option-drag each of the drives to that folder,
creating aliases. I'll bet dollars to doughnuts that GC will handle
the aliases if you just start your search from that container folder.

On 3/23/07 11:43 AM, "Power Donald" <depowerroadrunner.com> wrote:

> On Mar 22, 2007, at 2:21 PM, Tom Robinson wrote:
>
>> You missed my unspoken assumption there would be a manual high-level
>> combine of the drives first ;-)
>
> I, too, missed your unspoken assumption about a manual high-level
> combine of drives.
> HOW CAN a senior citizen (born before 1935) learn how to make a
> "manual combine of multiple drives"?

If you have drive space, that's as simple as making a folder for each source
drive and dumping it into your master drive.

On the other hand, if this is purely for images (and it's beginning to sound
as though it is), you could use a photo album program like iView Multimedia
and index all your photos on their individual drives. Then by sorting by
name and date, like photos should be awful close to one another. Delete
duplicates from the photo index, and then export all the indexed photos to
your new master drive.

A bit more manual, but a fair sight faster than combing through completely
by hand.

--
Nik :: gerberiNik.net
Make a developer cry! Vote for the top Mac software ever!
<http://www.squidoo.com/topmacsoftware/>

>>> 3.3.3.2 Browser Find duplicate files
>>> This functions searches for duplicate files starting from the
>>> current viewed folder in the browser.
>
> And there's the rub, no? That doesn't allow you to find duplicates
> across multiple drives, but just in a single directory tree.
>
> Or am I missing something?

As far as I know, all the disks on your system are in a single directory
tree.

I don't have a Mac to hand at the moment, but isn't there a directory
/Volumes which contains all your mounted volumes? Point GraphicConverter
at that folder, and I'd expect it to start searching across all volumes
except the startup volume. You'd have to start at / to get all volumes,
I don't know if you can point GraphicConverter at / , though.

Harro de Jong

At 12:55 PM -0700 3/24/07, Nik wrote:
>On the other hand, if this is purely for images (and it's beginning to sound
>as though it is), you could use a photo album program like iView Multimedia
>and index all your photos on their individual drives. Then by sorting by
>name and date, like photos should be awful close to one another. Delete
>duplicates from the photo index, and then export all the indexed photos to
>your new master drive.

iView Media Pro can also identify duplicates, and would enable the
creation of a catalog across multiple (online) drives.

cheers... -Adam

On Mar 26, 2007, at 11:29 AM, Harro de Jong wrote:
> As far as I know, all the disks on your system are in a single
> directory
> tree.
>
> I don't have a Mac to hand at the moment, but isn't there a directory
> /Volumes which contains all your mounted volumes? Point
> GraphicConverter
> at that folder, and I'd expect it to start searching across all
> volumes
> except the startup volume. You'd have to start at / to get all
> volumes,
> I don't know if you can point GraphicConverter at / , though.

Maybe I'm missing something, but I know of no "directory tree"
containing "/Volumes" in Macintosh usage. If there is such an entity,
my apologies. I also discovered that Graphic Converter does NOT
search in aliases (at least, I could not get it to search in aliases
in my experimentation).
Don

[Sure. Choose Go > Go to Folder in the Finder, and type /Volumes and hit Return. -Adam]