Sponsored in part by... Bare Bones Software Yojimbo 1.5 from Bare Bones Software: Your effortless, reliable
information organizer for Mac OS X. It will change your life,
without changing the way you work. Download the demo or buy it
today! <http://www.barebones.com/products/yojimbo/>

 [F] TidBITS  / TidBITS  / TidBITS Talk  /

Nits related to Unix in Tiger

[schinder]schinder (apparently) - 08:03pm May 2, 2005 PST
via email

I've been running Tiger for a few days now on my G5 and have run into
the following little annoying problems with the Unix side.

1) Thanks, Apple, for telling me that /rsrc is now a deprecated way of
getting information about the resource fork from the command line. But
is it really necessary to tell me each and every time any program tries
to do that? I'm doing a backup with RsyncX, and /var/log/system.log is
filling up with endless repetitions that "May 2 16:11:54 G5 kernel[0]:
HFS: /rsrc paths are deprecated (..namedfork/rsrc)". This is slowing
down both my backup and my machine. I'm going to change my kernel
syslog level in syslog.conf (if I can safely do it, is this really a
level 0 message?), but that will merely keep the messages out of my
logs, not keep them from being generated and sent to syslog. The
information could have simply been put in a README somewhere.

2) I'm sure ImageIO is a spiffy piece of software, but if you're going
to use a name like libJPEG.dylib in (I'll break this path in the middle
so it fits on a page)
/System/Library/Frameworks/ApplicationServices.framework/Versions/
A/Frameworks/ImageIO.framework/Versions/A/Resources/
would you mind doing it on a case sensitive file system? I get lots of

g5% /usr/local/bin/gnuplot
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from:
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /sw/lib/libjpeg.62.dylib

I know next to nothing about frameworks, but given the name, why don't
the framework directories get searched before the directories in
DYLD_LIBRARY_PATH? A cursory glance at libJPEG.dylib with nm doesn't
show much in common with the JPEG group's libjpeg. I can work around
the problem (it's showed up with more than one program) with little
shell scripts in my personal ~/bin like ~/bin/gnuplot:

#!/bin/sh
DYLD_LIBRARY_PATH=/usr/lib:/usr/X11R6/lib
export DYLD_LIBRARY_PATH
exec /usr/local/bin/gnuplot $*

to keep it from finding /sw/lib/libjpeg, but that's pretty annoying.

(Now, can anyone tell me why Dan Bernstein's supervise needs to chew up
CPU time in Tiger? When I boot I have to immediately su and kill off
svscan and all the supervise's to get the load down from 8. qmail and
djbdns and the multilog's all run fine and function properly, but
something screwy is going on with supervise. I suspect it has something
to do with launchd, but I haven't figured out what yet.)

--
Paul Schinder
schinderpobox.com


Mark as Read
  OutlineAll MessagesOlder MessagesOldest MessagesNewest MessagesNewer Messages

jwblist (apparently) - May 3, 2005 11:26 am (#1 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 768
Re: Nits related to Unix in Tiger

On 5/2/2005 20:03, "Paul Schinder" <schinderpobox.com> wrote:

> (Now, can anyone tell me why Dan Bernstein's supervise needs to chew up
> CPU time in Tiger? When I boot I have to immediately su and kill off
> svscan and all the supervise's to get the load down from 8. qmail and
> djbdns and the multilog's all run fine and function properly, but
> something screwy is going on with supervise. I suspect it has something
> to do with launchd, but I haven't figured out what yet.)

I hope this has to do with Tiger Server. I absolutely will not attempt to
run qmail.

  --John



atlauren (apparently) - May 3, 2005 11:26 am (#2 Total: 13)  

Reply to this message
via email - Practicing random acts of punditry.  

Photo of Author
Posts: 814
Re: Nits related to Unix in Tiger

At 8:03 PM -0700 5/2/05, Paul Schinder wrote:
>I'm doing a backup with RsyncX, and /var/log/system.log is
>filling up with endless repetitions that "May 2 16:11:54 G5 kernel[0]:
>HFS: /rsrc paths are deprecated (..namedfork/rsrc)".

In Tiger you need to do rsync -E to get the extended data bits (aka
resource forks).

--
Andrew Laurence
atlaurenuci.edu

John C. Welch (apparently) - May 3, 2005 11:26 am (#3 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 858
Re: Nits related to Unix in Tiger

On 5/2/05 22:03, "Paul Schinder" <schinderpobox.com> wrote:

> 2) I'm sure ImageIO is a spiffy piece of software, but if you're going
> to use a name like libJPEG.dylib in (I'll break this path in the middle
> so it fits on a page)
> /System/Library/Frameworks/ApplicationServices.framework/Versions/
> A/Frameworks/ImageIO.framework/Versions/A/Resources/
> would you mind doing it on a case sensitive file system? I get lots of

There's also the idea that software should stop making assumptions about
what it's getting installed into.

--
John C. Welch Writer/Analyst
Bynkii.com Mac and other opinions
jwelchbynkii.com



Chris Pepper (apparently) - May 3, 2005 1:03 pm (#4 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 845
At 11:26 AM -0700 2005/05/03, Andrew Laurence wrote:
>At 8:03 PM -0700 5/2/05, Paul Schinder wrote:
>>I'm doing a backup with RsyncX, and /var/log/system.log is
>>filling up with endless repetitions that "May 2 16:11:54 G5 kernel[0]:
>>HFS: /rsrc paths are deprecated (..namedfork/rsrc)".
>
>In Tiger you need to do rsync -E to get the extended data bits (aka
>resource forks).

        Dude! That's excellent. I've been hoping this would ship in
Tiger so long, I'd forgotten about it. I don't see any mention of
forks in the cp/mv/rm manual pages, though.

        Do you know if their patches are compatible with RsyncX
<http://archive.macosxlabs.org/rsyncx/rsyncx.html>?


                                                Chris Pepper
--
Chris Pepper: <http://www.reppep.com/~pepper/>
Rockefeller University: <http://www.rockefeller.edu/>

Andrew Laurence - May 3, 2005 1:03 pm (#5 Total: 13)  

Reply to this message
Guest User  

Photo of Author
Posts: 1
At 11:51 AM 5/3/2005, Chris Pepper wrote:
>At 11:26 AM -0700 2005/05/03, Andrew Laurence wrote:
>>At 8:03 PM -0700 5/2/05, Paul Schinder wrote:
>>>I'm doing a backup with RsyncX, and /var/log/system.log is
>>>filling up with endless repetitions that "May 2 16:11:54 G5 kernel[0]:
>>>HFS: /rsrc paths are deprecated (..namedfork/rsrc)".
>>
>>In Tiger you need to do rsync -E to get the extended data bits (aka
>>resource forks).
>
> Dude! That's excellent. I've been hoping this would ship in Tiger
> so long, I'd forgotten about it. I don't see any mention of forks in the
> cp/mv/rm manual pages, though.

At Apple's USENIX presentation, they mentioned several times that the
"usual file handling commands" (cp, mv, rm, etc) now all handle resource
forks. Apparently they just do it. rsync was mentioned as the (only?)
exception where a flag was required.

Tiger ships with rsync 2.6.3. I would imagine they sent their changes to
Tridge, but don't know one way or another.

> Do you know if their patches are compatible with RsyncX
> <http://archive.macosxlabs.org/rsyncx/rsyncx.html>?

No idea.

-Andrew


kevinv (apparently) - May 3, 2005 4:13 pm (#6 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 1399
Quoting Chris Pepper <pepperreppep.com>:

>> In Tiger you need to do rsync -E to get the extended data bits (aka
>> resource forks).
>
> Dude! That's excellent. I've been hoping this would ship in Tiger so
> long, I'd forgotten about it. I don't see any mention of forks in the
> cp/mv/rm manual pages, though.

Does this have to be supported on the server side too, or can i use
rsync -E to
a generic linux box?

Kevin


Chris Pepper (apparently) - May 3, 2005 4:13 pm (#7 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 845
At 1:03 PM -0700 2005/05/03, Andrew Laurence wrote:
>At 11:51 AM 5/3/2005, Chris Pepper wrote:
>>At 11:26 AM -0700 2005/05/03, Andrew Laurence wrote:

>>>In Tiger you need to do rsync -E to get the extended data bits (aka
>>>resource forks).
>>
>> Dude! That's excellent. I've been hoping this would ship in Tiger
>>so long, I'd forgotten about it. I don't see any mention of forks in the
>>cp/mv/rm manual pages, though.
>
>At Apple's USENIX presentation, they mentioned several times that the
>"usual file handling commands" (cp, mv, rm, etc) now all handle resource
>forks. Apparently they just do it. rsync was mentioned as the (only?)
>exception where a flag was required.

<http://www.apple.com/macosx/features/unix/>

>HFS+ CLI file commands
>Use command-line commands safely on HFS+ files. Utilities such as
>cp, mv, tar, rsync now use the same standard APIs as Spotlight and
>access control lists to handle resource forks properly.


                                                Chris
--
Chris Pepper: <http://www.reppep.com/~pepper/>
Rockefeller University: <http://www.rockefeller.edu/>

lists748 (apparently) - May 3, 2005 4:13 pm (#8 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 8
On May 3, 2005, at 4:03 PM, Andrew Laurence wrote:

> At Apple's USENIX presentation, they mentioned several times that the
> "usual file handling commands" (cp, mv, rm, etc) now all handle
> resource
> forks. Apparently they just do it. rsync was mentioned as the
> (only?)
> exception where a flag was required.

scp also requires the -E flag.

--Michael

Chris Pepper (apparently) - May 4, 2005 10:26 am (#9 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 845
Re: Nits related to Unix in Tiger

At 4:13 PM -0700 2005/05/03, kevinvanhaaren.net wrote:
>Quoting Chris Pepper <pepperreppep.com>:
>
>>>In Tiger you need to do rsync -E to get the extended data bits (aka
>>>resource forks).
>>
>> Dude! That's excellent. I've been hoping this would ship in Tiger so
>>long, I'd forgotten about it. I don't see any mention of forks in the
>>cp/mv/rm manual pages, though.
>
>Does this have to be supported on the server side too, or can i use
>rsync -E to
>a generic linux box?

        The note is unclear, but I think it requires patches on both sides:

> -E, --extended-attributes
> Apple specific option to copy extended attributes, resource
> forks, and ACLs. Requires at least Mac OS X 10.4 or suitably
> patched rsync.

        Otherwise there's no real point in telling you that the rsync
included in 10.4 requires 10.4...


At 4:13 PM -0700 2005/05/03, Michael Tsai wrote:

>scp also requires the -E flag.

        But there's no -E in the sftp manual page. This is a
nuisance, although I normally use scp anyway (except when going to an
ssh.com ssh server, since ssh.com's Protocol 2 scp is incompatible
with OpenSSH's scp -- fortunately I think we're going to start
getting rid of ssh.com).

        Can anyone here comment on resource/metadata support in sftp?


                                                Chris
--
Chris Pepper: <http://www.reppep.com/~pepper/>
Rockefeller University: <http://www.rockefeller.edu/>

atlauren (apparently) - May 4, 2005 12:40 pm (#10 Total: 13)  

Reply to this message
via email - Practicing random acts of punditry.  

Photo of Author
Posts: 814
At 10:26 AM -0700 5/4/05, Chris Pepper wrote:
> Can anyone here comment on resource/metadata support in sftp?

Offlist, I asked Apple's Ernest Prabhakar to look at this thread...

At 11:12 AM -0700 5/4/05, Ernest Prabhakar wrote:
>The complete list is in the UNIX Tech Brief:
>
>http://images.apple.com/macosx/pdf/MacOSX_UNIX_TB.pdf
>
>* cp, mv, scp, emacs, vim, pico. Properly handle HFS+ resource forks
>using the new extended attribute APIs, as do archivers such as tar,
>rsync, gzip, bzip2, and cpio.
>
>They are correct that rsync and scp require the '-E' flag, as listed
>in the man page, because the current protocol doesn't allow resource
>forks, and that DOES make them incompatible with existing servers.
>We didn't even want to try messing with sftp...
>
>Hope this helps. Feel free to pass this information along, as I
>don't like to post to lists I don't monitor regularly.
>
>-- Ernie P.

--
Andrew Laurence
atlaurenuci.edu

Curtis Wilcox (apparently) - May 5, 2005 9:58 am (#11 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 357
Re: Nits related to Unix in Tiger

> -----Original Message-----
> From: Chris Pepper [mailto:pepperreppep.com]
> Sent: Wednesday, May 04, 2005 1:27 PM
> To: tidbits-talktidbits.com
> Cc: tidbits-talktidbits.com
> Subject: Re: Nits related to Unix in Tiger

> >Does this have to be supported on the server side too, or can i use
> >rsync -E to a generic linux box?
>
> The note is unclear, but I think it requires patches on
> both sides:
>
> > -E, --extended-attributes
> > Apple specific option to copy extended attributes, resource
> > forks, and ACLs. Requires at least Mac OS X 10.4 or suitably
> > patched rsync.

I'm sure it requires the same patches on both sides, that's the way the
hfs-aware version of rsync distributed with RsyncX has always been. Speaking
of which, I doubt you can use RsyncX's rsync on one side and 10.4's rsync on
the other to successfully copy resource forks.

Of course the Linux server would also have to know what to do with the
resource forks, ACLs, etc. once they arrived.

> Otherwise there's no real point in telling you that the rsync
> included in 10.4 requires 10.4...

I tried copying 10.4's rsync program to a OS X 10.2 computer but it fails to
run because of the lack of a library (libresolv-something-or-other).

The source code for 10.4's rsync is available for download but I don't know
that this is sufficient to compile a version of rsync with the extended
attributes capabilities. I took a stab at compiling it (on OS X 10.2) but I
didn't figure out how to incorporate the changes, I kept ending up with
vanilla rsync.
http://www.opensource.apple.com/darwinsource/10.4/rsync-20/

Chris Pepper (apparently) - May 5, 2005 9:58 am (#12 Total: 13)  

Reply to this message
via email  

Photo of Author
Posts: 845
Re: Nits related to Unix in Tiger

At 10:28 AM -0400 2005/05/05, Wilcox, Curtis wrote:
>The source code for 10.4's rsync is available for download but I don't know
>that this is sufficient to compile a version of rsync with the extended
>attributes capabilities. I took a stab at compiling it (on OS X 10.2) but I
>didn't figure out how to incorporate the changes, I kept ending up with
>vanilla rsync.
>http://www.opensource.apple.com/darwinsource/10.4/rsync-20/

Curtis,
        <http://www.opensource.apple.com/darwinsource/10.4/rsync-20/patches/EA.diff>
says:

>+# check for extended attribute support
>+echo "$as_me:$LINENO: checking whether to support extended attributes" >&5
>+echo $ECHO_N "checking whether to support extended attributes... $ECHO_C" >&6
>+# Check whether --enable-ea-support or --disable-ea-support was given.
>+if test "${enable_ea_support+set}" = set; then
>+ enableval="$enable_ea_support"

        Did you use "configure --enable-ea-support"?

        To build it the normal UNIX way, I belive you should download
three files (stripping any .txt suffixes that may be added):

http://www.opensource.apple.com/darwinsource/10.4/rsync-20/rsync-2.6.3.tar.gz
http://www.opensource.apple.com/darwinsource/10.4/rsync-20/patches/EA.diff
http://www.opensource.apple.com/darwinsource/10.4/rsync-20/patches/PR-3945747-endian.diff

        Copy all three files into a new directory (perhaps rsync.ea),
and from that directory try something like the following:

tar xzf rsync-2.6.3.tar.gz # extract the base rsync source
cd rsync-2.6.3
patch < ../EA.diff # Apply extended attribute patch
patch < ../PR-3945747-endian.diff # apply endian patch
--configure --enable-ea-support # configure with extended attribute support
make # this fails for me on line of sender.c.


        Apple's Darwin build system is totally different. It looks
like the Makefile they supply should do everything automatically,
including unpacking and patching the base tarball (good for an OS
vendor, both for repeatability and scaling), but that didn't work
either when I tried. I don't know why -- perhaps I'd need to install
other bits of the Darwin build system first.


                                                Chris
--
Chris Pepper: <http://www.reppep.com/~pepper/>
Rockefeller University: <http://www.rockefeller.edu/>

Aaron Iba - May 10, 2005 8:26 am (#13 Total: 13)  

Reply to this message
 

Photo of Author
Posts: 1
Re: Nits related to Unix in Tiger

I too have been using RSyncX to do backups. On tiger, when I use /usr/bin/rsync -E, I get the following errors:

   
building file list ... rsync: writefd_unbuffered failed to write 4092 bytes: phase "send_file_entry" [sender]: Broken pipe (32)   
overflow: flags=0x726f l1=101 l2=12292 lastname=._Desktop   
ERROR: buffer overflow in receive_file_entry   
rsync error: error allocating core memory buffers (code 22) at /SourceCache/rsync/rsync-20/rsync/util.c(126)   
rsync: connection unexpectedly closed (8 bytes received so far) [sender]   
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-20/rsync/io.c(359)   


Has anyone seen errors like this? I looked at the source code to rsync-20, and it seems that it is reading bunk data from the file descriptor, so I thought maybe my filesystem is corrupted somehow (although Disk Utilities "verifies" it successfully).

Any ideas what this is?



  OutlineAll MessagesOlder MessagesOldest MessagesNewest MessagesNewer Messages


 [F] TidBITS  / TidBITS  / TidBITS Talk  / Nits related to Unix in Tiger




Add a message

To add a message to this discussion, you must be a registered user. Enter your email address below. If you have an account associated with the email address you enter, you will be prompted for your password. If not, you'll be able to create a new account with no fuss.

Enter your email address:

Submit