Nick Shubin
 Home  Articles  Photos  Publications  Site Map

 

Nick Shubin

Why Files are Much Larger than Expected

on Macintosh computers

March 19, 2010


I used to have HTML files translated to different languages. The received files had one major shortcoming. They were too large. While the originals were about 8–16 KB, the translations appear to be about 64 KB. You may say that extra 40 kilobytes isn't something worth to worry about. Not in my case – it was a Help system containing dozens HTML files. The lengths of the original and translated texts didn't differ so much.

The second problem was connected with copying these files to a shared network disk. Although I had Read & Write privilege set, copying them was impossible. In the error message, there was something about write permissions. The problem occurred only with those translations.

A solution came from our sysadmin. He helped to fix the second problem what appeared to be a solution for the first one too. I was suggested to check the extended attributes of my files. This can be done using the shell command "ls -la@".

A command line example:
ls -la@ /Volumes/data/Pages001-100

The output may look as the following:
-rwxr-xr-x@ 1 nick staff 4282 Aug 10 2009 p001.html
com.apple.FinderInfo 32
com.apple.quarantine 70
-rw-r-----@ 1 nick staff 5637 Aug 10 2009 p002.html
com.apple.FinderInfo 32
com.apple.quarantine 70
-rw-r--r--@ 1 nick staff 26741 Sep 29 16:38 img01.jpg
com.apple.FinderInfo 32
com.apple.ResourceFork 29579
com.apple.quarantine 70
-rw-r--r--@ 1 nick staff 4512 Jul 8 2009 img02.gif
com.apple.FinderInfo 32
com.apple.ResourceFork 55873
com.apple.metadata:kMDItemFinderComment 174
com.apple.quarantine 70

The symbol @ in the attributes block indicates that a file (its name is in the end of the line) has extended file attributes. We have:
com.apple.FinderInfo
com.apple.quarantine
com.apple.ResourceFork
com.apple.metadata:kMDItemFinderComment

Note that the number near the attribute name is its size. In the example, com.apple.FinderInfo, com.apple.quarantine and com.apple.metadata:kMDItemFinderComment don't take much space. But look at com.apple.ResourceFork. A resource fork file is larger than the file it accompanies. In case of img02.gif, the resource fork is 12 times larger than the data fork (actual graphic file)!

Now we came up to actions. It is possible to examine the metadata or even remove it. Note that by deleting metadata you can prevent some software from treating your document(s) correctly. If you wish to experiment, back up your file(s).

The xattr utility can be used to list and remove extended file and folder attributes. To list attributes, use this command with the path to a file:
xattr /Volumes/data/Pages001-100/img02.gif

An amateur hacker may wish to explore the content of the resource forks. The -l key lets you do that:
xattr -l /Volumes/data/Pages001-100/img02.gif

In the output, you will find data in binary format as shown below:
com.apple.FinderInfo:
0000 47 49 46 66 38 42 49 4D 04 00 00 00 00 00 00 00 GIFf8BIM........
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

com.apple.ResourceFork:
0000 00 00 01 00 00 00 D4 4A 00 00 D3 4A 00 00 02 BA .......J...J....
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

... tons of lines (actually here are 55873 bytes of data) containing the name of the program-creator, modification time, etc.

com.apple.metadata:kMDItemFinderComment:
0000 62 70 6C 69 73 74 30 30 5F 10 82 38 2D 62 69 74 bplist00_..8-bit
0010 20 67 72 61 79 73 63 61 6C 65 20 66 6C 61 74 20 grayscale flat
0020 43 6F 6D 70 75 53 65 72 76 65 20 47 49 46 20 66 CompuServe GIF f
0030 69 6C 65 2C 20 31 30 34 39 78 35 38 34 20 70 69 ile, 1049x584 pi
0040 78 65 6C 73 20 28 33 2E 35 30 78 31 2E 39 35 20 xels (3.50x1.95
0050 69 6E 63 68 65 73 29 20 40 20 33 30 30 2E 30 30 inches) @ 300.00
0060 20 70 69 78 65 6C 73 2F 69 6E 63 68 2C 20 77 72 pixels/inch, wr
0070 69 74 74 65 6E 20 62 79 20 41 64 6F 62 65 20 50 itten by Adobe P
0080 68 6F 74 6F 73 68 6F 70 20 43 53 33 20 08 00 00 hotoshop CS3 ...
0090 00 00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 ................
00A0 00 00 00 00 00 00 00 00 00 00 00 00 00 8D       ..............

com.apple.quarantine:
0000;4b82815f;Mail;17D85494-83F5-46B1-822C-7FA0AF013A83|com.apple.mail

I have to note that the actual image dimensions are 400x223 pixels @ 300dpi what differs from the information in com.apple.metadata:kMDItemFinderComment.

To delete an extended attribute, use xattr with the -d key:
xattr -d com.apple.quarantine /Volumes/data/Pages001-100/img02.gif

In my example, removing the extended attributes of img02.gif has changed its size from 64 KB (60385 bytes) to 8 KB (5412 bytes).

You can replace file names with wildcards. This command removes the quarantine attribute of all files in the current folder:
xattr -d com.apple.quarantine *

It seems that xattr doesn't accept wildcards for the attribute name.

 

Related information:

Resource fork

Filesystem metadata support


© 2016 Nick Shubin. All rights reserved.