Time Capsule and Its Associated Rage Factor

Further associated rage comes from the pricing of the new consumer device. It's damned cheap. I think that's great. However, it puts Apple in a fairly interesting position when comparing to the upgrade pricing of the brand new Mac Pro's. It costs $550 to add a 1TB hard drive to a Mac Pro, but $500 for a 1TB Time Capsule? We recently spec'd out a Mac Pro for video editing. I pointed out to our Apple rep that we could just order 3 Time Capsules and pull the "server-grade" hard drives out and throw them in the Mac Pro for cheaper.. Then we'd have a few Airport Extreme base stations to play with.. with room for expansion. If I worked for a smaller company that didn't have to track assets, I'd actually consider this.

Of course, I like our Apple rep, so I was just jabbing him... I know he doesn't control the pricing.. but really.. the markup on those drives is just plain unreasonable. I realize we could get 3rd party drives elsewhere, but we'd rather have the official support and warranty from Apple.

I bet they won't sell many 1TB drives outside of the Time Capsule.

On Jan 18, 2008 8:03 AM, <brett_xmac.com> wrote:
> Of course, I like our Apple rep, so I was just jabbing him... I know he doesn't control the pricing.. but really.. the markup on those drives is just plain unreasonable. I realize we could get 3rd party drives elsewhere, but we'd rather have the official support and warranty from Apple.

Likewise, Apple's markup on RAM is ludicrous.

In the case of the Mac Pro, both hard drives and RAM are
user-installable-parts, so you shouldn't hurt on warranty. At worst,
you'd have to replace the drive under the manufacturer's warranty. For
larger businesses, that's not a real hardship, since you can afford to
have a few spares ready in case of failure.

--Nik

Could it be that drive write speed is a factor in the decision to allow
Time Machine to Time Capsule, but not an Airport Extreme base station? I
wouldn't want to back up anything to a drive connected to my Extreme,
it's something like 10x slower than writing to a local disk.

Then again, Time Capsule is said to use the same hardware, which would
make speed improvements unlikely.

Harro de Jong

On 21-Jan-2008, at 15:14, Harro de Jong wrote:
> Could it be that drive write speed is a factor in the decision to
> allow
> Time Machine to Time Capsule, but not an Airport Extreme base station?

No. I think it was simply that there was no way to schedule the
writes to a USB drive so as to prevent issues when multiple clients
would try to write to the drive (like if two TM backups ran at once).
Also, there is no way to be sure that a USB drive has actually emptied
its cache. I think that the USB HD issues are what kept Apple from
shipping Leopard with AE support for Time Machine and got them to
release the Time Capsule product.

> Then again, Time Capsule is said to use the same hardware, which would
> make speed improvements unlikely.

The drive in the Time Capsule is not a USB drive, that right there
will yeild significant speed boosts.

Glenn Fleishman wrote:
> "But if Apple can now reliably write backups to a hard drive connected via Serial ATA, why can't it handle a drive connected via USB?"

Because Apple bought the Time Capsule drive themselves.

First, USB is not a native protocol for hard drives. Any USB drive is
actually a SATA or PATA disk inside, with a bridge connecting the disk
to the USB bus. Second, each layer of control adds a cache. These
caches improve speed significantly. But caches are lost when power is
lost unexpectedly, so software that requires 100% reliability issues
frequent cache flush commands.

With external USB cases, most manufacturers ignore these cache flush
commands to improve their benchmark scores! In fact, even some hard
drives do this. So with external USB disks, there can be two layers
lying about data integrity.

With Time Capsule, Apple can ensure the drive they use conforms to the
standards properly. There's also no unknown protocol bridge between
the CPU and the drive. And that's why Apple can reliably write backups
to a directly connected SATA hard drive but not an external USB one.

-pmb

At 6:26 AM -0800 1/23/08, Peter Bierman wrote:
>Glenn Fleishman wrote:
>> "But if Apple can now reliably write backups to a hard drive
>>connected via Serial ATA, why can't it handle a drive connected via
>>USB?"
>
>Because Apple bought the Time Capsule drive themselves.
>
>First, USB is not a native protocol for hard drives. Any USB drive is
>actually a SATA or PATA disk inside, with a bridge connecting the disk
>to the USB bus. Second, each layer of control adds a cache. These
>caches improve speed significantly. But caches are lost when power is
>lost unexpectedly, so software that requires 100% reliability issues
>frequent cache flush commands.
>
>With external USB cases, most manufacturers ignore these cache flush
>commands to improve their benchmark scores! In fact, even some hard
>drives do this. So with external USB disks, there can be two layers
>lying about data integrity.
>
>With Time Capsule, Apple can ensure the drive they use conforms to the
>standards properly. There's also no unknown protocol bridge between
>the CPU and the drive. And that's why Apple can reliably write backups
>to a directly connected SATA hard drive but not an external USB one.

It seems to me that everything you wrote would apply to a USB drive
connected directly to your Mac (rather than through an AEBS) - and
yet Time Machine works with direct connect drives.

Why do you suppose that is?
--
-- Marshall

On Jan 23, 2008, at 9:26 AM, Peter Bierman wrote:

> With Time Capsule, Apple can ensure the drive they use conforms to the
> standards properly. There's also no unknown protocol bridge between
> the CPU and the drive. And that's why Apple can reliably write backups
> to a directly connected SATA hard drive but not an external USB one.

Sure, but if Apple were concerned about that, why would they enable
Time Machine for USB drives connected directly to a Mac?

--Michael

Peter Bierman wrote:
> Glenn Fleishman wrote:
>> "But if Apple can now reliably write backups to a hard drive connected via Serial ATA, why can't it handle a drive connected via USB?"
>
> Because Apple bought the Time Capsule drive themselves.
>
> First, USB is not a native protocol for hard drives. Any USB drive is
> actually a SATA or PATA disk inside, with a bridge connecting the disk
> to the USB bus. Second, each layer of control adds a cache. These
> caches improve speed significantly. But caches are lost when power is
> lost unexpectedly, so software that requires 100% reliability issues
> frequent cache flush commands.
>
> With external USB cases, most manufacturers ignore these cache flush
> commands to improve their benchmark scores! In fact, even some hard
> drives do this. So with external USB disks, there can be two layers
> lying about data integrity.
>
> With Time Capsule, Apple can ensure the drive they use conforms to the
> standards properly. There's also no unknown protocol bridge between
> the CPU and the drive. And that's why Apple can reliably write backups
> to a directly connected SATA hard drive but not an external USB one.

So basically you're saying that USB drives (and I assume Firewire?)
shouldn't be used for serious purposes. Why is a backup any more
important than anything else I might put on an external drive?

Fortunately for me, I guess, only my backup drives are Firewire. The
others are SATA or ATA.

--
Paul Schinder
schinderpobox.com

Paul,

I don't think that was what Marshall was saying. He was simply
pointing out your error in assuming the existence of USB and Firewire
drives. As Marshall pointed out, there are no such animals. They do
not exist. Drives are sata,pata, or scsi. Period. When installed into
external enclosures, the bridge cards convert the interfaces to USB or
Firewire.

The rest about the write caching has been true for at least ten years
that I personally know of. I used to work as a senior tech at CMD
Technologies (a pretty big RAID system manufacturer in its day) and
had to deal with those issues all of the time.

The only time it is really and truly an issue is if you have a failure
during the time you save data to the drive. Once the data is written
to the cache, the drive tells the host it is written. If the drive
fails before it actually moves it from the cache to the media of the
drive is when it causes problems.

The probabilities of this actually happening are REALLY low if you are
looking at failures. It is more likely to have an issue if you force
unmount the drive before it finishes writing.

Jeff Pearson


On Jan 23, 2008, at 10:52 AM, Paul Schinder wrote:

> So basically you're saying that USB drives (and I assume Firewire?)
> shouldn't be used for serious purposes. Why is a backup any more
> important than anything else I might put on an external drive?
>
> Fortunately for me, I guess, only my backup drives are Firewire. The
> others are SATA or ATA.

At 7:07 PM -0800 1/23/08, Jeffrey Pearson wrote:
>On Jan 23, 2008, at 10:52 AM, Paul Schinder wrote:
>
>>So basically you're saying that USB drives (and I assume Firewire?)
>>shouldn't be used for serious purposes. Why is a backup any more
>>important than anything else I might put on an external drive?
>>
>>Fortunately for me, I guess, only my backup drives are Firewire. The
>>others are SATA or ATA.
>
>I don't think that was what Marshall was saying. He was simply
>pointing out your error in assuming the existence of USB and Firewire
>drives. As Marshall pointed out, there are no such animals. They do
>not exist. Drives are sata,pata, or scsi. Period. When installed into
>external enclosures, the bridge cards convert the interfaces to USB or
>Firewire.

I think that people are misunderstanding what I wrote (or maybe I was
not clear).

Peter Bierman opined that the reason that Apple did not support Time
Machine backups
on USB drives connected to AEBS was the worry that data would be
cached on the drive, (or possibly in the USB/ATA or USB/SATA bridge
circuitry) and possibly not written in case of a power failure.
[ For his exact words, see his email of Jan 23 at 6:26 AM PST ]

I merely pointed out that drives in external cases connected directly
to a Mac (rather than connected to a base station) had the exact same
failure mode, and asked why Apple supported Time Machine backups in
that case.

It was not a general polemic on USB drives.
--
-- Marshall

I find this discussion a bit strange. Maybe we should wait for the
next update to Mac OS X and see whether wireless backups to "normal"
AirPort stations work before we try to find excuses explaining why
Apple could not possibly ever implement such an advanced feature
(which works just fine for backing up my MacBook Pro wirelessly to USB
disks connected to my Mac mini).

Jeffrey Pearson wrote:
> Paul,
>
> I don't think that was what Marshall was saying. He was simply
> pointing out your error in assuming the existence of USB and Firewire
> drives. As Marshall pointed out, there are no such animals. They do
> not exist. Drives are sata,pata, or scsi. Period. When installed into
> external enclosures, the bridge cards convert the interfaces to USB or
> Firewire.

That I knew.

>
> The rest about the write caching has been true for at least ten years
> that I personally know of. I used to work as a senior tech at CMD
> Technologies (a pretty big RAID system manufacturer in its day) and
> had to deal with those issues all of the time.

That I didn't. Ignoring cache flush? Doesn't that mean that every time
you shut down or reboot, you run the risk of data corruption? Or is
there an "I'm shutting down now, so cache flush and I really mean it"
signal that goes out on the bus? Any particular manufacturers that
ignore cache flush, or do they all do it?

> The only time it is really and truly an issue is if you have a failure
> during the time you save data to the drive. Once the data is written
> to the cache, the drive tells the host it is written. If the drive
> fails before it actually moves it from the cache to the media of the
> drive is when it causes problems.
>
> The probabilities of this actually happening are REALLY low if you are
> looking at failures. It is more likely to have an issue if you force
> unmount the drive before it finishes writing.

I understand the risks are low. If so, it makes little sense for Apple
to prevent the use of certain external AFP mounts from being used as
Time Machine backups, whatever it happens to be connected to. I know
that Apple is keen on user experience, and may have rationalizations for
this behavior. But this is one case where there's no moral dilemma, and
if I had an Extreme and an external USB drive, I'd be hunting for ways
to get around whatever Apple has done. After all, they don't prevent
you from using a drive connected to an Extreme (and other data is no
less important than Time Machine backups), they presumably can't prevent
third party backup software from using the drive, so why aren't they
allowing Time Machine to do it?

--
Paul Schinder
schinderpobox.com

On Jan 24, 2008, at 2:17 AM, Marshall Clow wrote:

> I think that people are misunderstanding what I wrote (or maybe I was
> not clear).
>
> I merely pointed out that drives in external cases connected directly
> to a Mac (rather than connected to a base station) had the exact same
> failure mode, and asked why Apple supported Time Machine backups in
> that case.
>
> It was not a general polemic on USB drives.

Marshall,

Yes. As you can tell from my response to you, I definitely
misunderstood the message you were trying to communicate.

I would actually trust a direct connect usb or firewire over any kind
of network accessed drive. I am a firm believer in the 'KISS'
philosophy. Any time you add layers/complexity, you add more potential
for issues.

I would also completely disagree with the opinion that that was why
Apple did not support TM backups on the AEBS was because of the "data
caching issue".

The majority (as in probably 99.9%)of 'manufacturers' creating
external fw/usb do not have any caching on the bridge cards. There is
no reason to. Caching is done to increase performance. The bottle-neck
of these drives are the usb/fw buses themselves...not the drives
themselves. The odds of a failure that happens when data has been
written but not stored on drive media is so extreme, it should not
even be being considered.

Think about it. Most drives these days have 8MB of CACHE, although
some have kicked that up to 16 MB. The thing they DON'T share is
whether that cache is dedicated write cache or is it shared as a read
cache as well. So assuming an even split (4MB of read cache, 4MB of
write cache), how long is the data (all 4MB of it) only existing in
cache? And you want to worry about a drive failing in this fraction of
a second? To me, it makes about as much sense worrying about this as
worrying about what I am going to spend all of the money on if I win
the lottery.

  It was not a technical decision but a business one, for whatever
reason.

Jeff Pearson

Nik Friedman TeBockhorst wrote:
> On Jan 18, 2008 8:03 AM, <brett_xmac.com> wrote:
>> Of course, I like our Apple rep, so I was just jabbing him... I
>> know he doesn't control the pricing.. but really.. the markup on
>> those drives is just plain unreasonable. I realize we could get 3rd
>> party drives elsewhere, but we'd rather have the official support
>> and warranty from Apple.
>
> Likewise, Apple's markup on RAM is ludicrous.
>
No. Apple's mark up on items you can buy from them or somewhere else are what pays for the nice stores, the genius bar, the ....

David Ross

On 01/25/2008 10:06 AM, "Lukas Mathis" <Lukas.Mthis.li> wrote:

> I find this discussion a bit strange. Maybe we should wait for the
> next update to Mac OS X and see whether wireless backups to "normal"
> AirPort stations work before we try to find excuses explaining why
> Apple could not possibly ever implement such an advanced feature
> (which works just fine for backing up my MacBook Pro wirelessly to USB
> disks connected to my Mac mini).

One thing I've not seen pointed out is that for Time Machine to really work
well, whatever is hosting the drive has to support directory hard links. If
you use a Mac OS X 10.5 Server - hosted AFP drive, not a problem. If you use
a direct-attached USB or FW drive, not a problem. However, if the previous
gen AEB didn't, then any USB-attached drives would have issues with that,
and therein could lay the problem.

--
John C. Welch Writer/Analyst
Bynkii.com Mac and other opinions
jwelchbynkii.com

On 1/25/2008 8:06 AM, "Jeffrey Pearson" wrote:
> It was not a technical decision but a business one, for whatever
> reason.

Although it may turn out to be a business decision going forward, the word
on the street is that it was indeed technical issues that kept Apple from
initially supporting Time Machine backups on the AEBS.

On Jan 25, 2008, at 9:21 PM, Dan Frakes wrote:

> On 1/25/2008 8:06 AM, "Jeffrey Pearson" wrote:
>> It was not a technical decision but a business one, for whatever
>> reason.
>
> Although it may turn out to be a business decision going forward,
> the word
> on the street is that it was indeed technical issues that kept Apple
> from
> initially supporting Time Machine backups on the AEBS.

It doesn't feel like a business decision. Apple hasn't lost touch
with reality that badly (if it has, we'll all be using Windows or
Linux (or nothing) and players other than iPods in a few years.

A rational business decision would have been not to mention using
Airport Extreme with an attached drive in the previews of Leopard.

   --John

[...and then either defer AEBS revenue to future quarters, or charge $20 for an AEBS update to provide the previously-unadvertised feature. Considering the iPod Touch upgrade fee, I wonder if we might start seeing more of the latter. -Andrew ]

From the user perspective, I don't give a R/A whether it was a technical issue or a business decision that is driving Apple's handling of network back-ups. What I care about is that I sank bucks in AEB and a 500GB USB drive in the expectation that that configuration would support regular system backups.

Now comes Time Capsule and speculation that Apple will abandon plans to support the AEB/USB HD backup method.

Seems to me, if Apple can't solve the AEB/USB HD issue, the preferred solution would be to offer AEB customers a $179 credit toward a trade-in of AEB for Time Capsule.

I might have to eat my existing USB expense (assuming I can't find another use for it), and add a few bucks to get what I need, but at least that would give me what I expected from AEB in the first place.

On 25-Jan-2008, at 22:21, Dan Frakes wrote:
> On 1/25/2008 8:06 AM, "Jeffrey Pearson" wrote:
>> It was not a technical decision but a business one, for whatever
>> reason.
>
> Although it may turn out to be a business decision going forward,
> the word
> on the street is that it was indeed technical issues that kept Apple
> from
> initially supporting Time Machine backups on the AEBS.

Yep. It simply did not work in the Leopard betas. Well, not reliably
enough to trust it.

On 25-Jan-2008, at 22:21, John C. Welch wrote:
> One thing I've not seen pointed out is that for Time Machine to
> really work
> well, whatever is hosting the drive has to support directory hard
> links.

Time Machine, when backing up over the network, uses a sparse file
disk image, not a straight up directory with hard links.

On 01/28/2008 10:56 AM, "LewisGmail" <gkremegmail.com> wrote:

> On 25-Jan-2008, at 22:21, John C. Welch wrote:
>> One thing I've not seen pointed out is that for Time Machine to
>> really work
>> well, whatever is hosting the drive has to support directory hard
>> links.
>
> Time Machine, when backing up over the network, uses a sparse file
> disk image, not a straight up directory with hard links.

Actually, it uses a sparse *bundle* which was introduced in Mac OS X 10.5.
Comparing the man pages for hdiutil in Mac OS X 10.4.11 and Mac OS X 10.5,
Mac OS X 10.4 can't create sparse bundles, just sparse images, so that's
another potential factor.

While you can make TM work with non- Mac OS X 10.5 - level OS's, via a well
known command, I do not see, from any level other than "all I care about is
up-front cost" why you would play games like that with *backups*. It's a
really bad idea on so many levels. Backups are not the thing to play fast
and loose with.

--
John C. Welch Writer/Analyst
Bynkii.com Mac and other opinions
jwelchbynkii.com