Last year around this time I was thinking about mp3 players. My trusty old Archos Jukebox 20 Studio just wasn’t cutting it anymore, even with its ROM flashed with open source Rockbox firmware. Yes, the Archos was a solid brick of an mp3 player, had a simple yellow LCD display, USB 1.1, and a very short battery life which required me to carry around its AC adapter wherever I went, but that’s not the reason I gave it up. I wanted “Album Shuffle”: the means for shuffling your songs by random album rather than random song. This is an important feature if you want to listen to any album that isn’t an 80s pop album with only one or two good songs on it, like for example, Vivaldi’s Four Seasons or Pink Floyd’s Wish You were Here. The order of tracks, representing movements or songs in a larger themed composition, matters. (I’ve written more about Album Shuffle here.)

Then I noticed that the ipods had album shuffle. The new players from Cowon and Archos did not, nor any others that fancied themselves as ipod competitors. But I still wasn’t convinced to buy an ipod yet. My trusty if heavy and slow Archos had a (then enormous) 100gb hard drive that I had installed myself and the largest ipod then available was 60gb. Ahh, but just before my birthday Apple announced their release of a new 160gb ipod. I was won over. Soon I gifted myself with a new Ipod Classic 160gb.

iPod Management

When it arrived, the ipod’s hard drive came formatted with Apple’s native file system, HFS Plus (HFS+). As the Windows operating systems cannot natively read HFS+ drives and my Thinkpad runs Windows XP, iTunes reformatted the ipod with FAT32, a file system engineered by Microsoft. At the time I didn’t think too much of HFS+ vs. FAT32, I was just happy that the ipod was working. And so, I put all concerns about file fragmentation and the need to periodically defrag FAT32 volumes to the side, and got to work filling the ipod up with good music and videos.

Over the last year I’ve learned how to corrupt my ipod’s database (and how to fix it painlessly) by avoiding iTunes as much as possible. iTunes had the advantage of supporting Album shuffle, but I preferred to use Winamp with the Album List plug-in for listening to albums on my computer. I had some success using Floola (which does not support Album shuffle) and Floola is my choice ipod manager on my Linux boxen. But on my Thinkpad running Windows XP, I was more interested in whether there were any plug-ins for Winamp that could suffice as a fully featured alternative to iTunes.

Looking at Winamp I discovered that it supported iPods through a plug-in bundled with the Winamp installer called pmp_ipod. Trying it out I was underwhelmed by its poor support of album cover art on the ipod, a feature I had really come to love. Then I discovered ml_ipod — an open source winamp plugin written by independent developers that could do (almost) everything pmp_ipod could do but better. The only thing I would need iTunes for would be for occasional firmware updates. ml_ipod support was fairly well documented on an online wiki and any further questions could be pursued on an active support forum hosted at Winamp. I’ve been using ml_ipod since January and have donated money to the further development of the plug-in.

File Fragmentation in FAT32 vs. HFS+

A few weeks ago I began to wonder again what my ipod’s FAT32 volume file fragmentation looked like. Unsurprisingly, after tens of thousands of file transfers, the ipod’s music, video, database and artwork files were critically fragmented according to Diskeeper, a windows defrag tool. A fragmented file system meant that my ipod needed to work harder and slower than it should have to. The answer to a fragmented ipod file system isn’t defragging it though. Ever wonder whether you should defrag your ipod? Don’t waste your time. Defragmenting an ipod over USB takes a LONG time. It is much much faster to simply do a full restore from your computer’s existing archive of music. (Before doing so, make sure you have an archive of all your iPod’s music.)

Even after I initialized and reloaded my FAT32 ipod, I found that the the iTunes database of music files as well as the artwork database of cover art were still fragmented — just less so. I began to explore what benefits there might be to manage the ipod with its original HFS+ over FAT32. I was impressed to find that HFS+ drives do not suffer from the same fragmentation problems as FAT32 drives. As this comparison of file systems shows, the main reason for the lack of fragmentation in HFS+ is because unlike FAT32, HFS+ supports Extents. Wikipedia explains:

An extent is a contiguous area of storage in a computer file system, reserved for a file. When starting to write to a file, a whole extent is allocated. When writing to the file again, possibly after doing other write operations, the data continues where the previous write left off. This reduces or eliminates file fragmentation.

Additionally, because HFS+ was specifically engineered to minimize disk access and quickly access individual files, its specific utility for the iPod seems obvious. This specific advantage of HFS+ over FAT32 was summarized well by the user, “Millenium,” over on the macnn web forum in a 2006 thread on HFS+ vs. FAT32:

You may hear that HFS+ is slower than FAT32. That’s true in some cases, but not in others. In particular, HFS+ does not do very well in tasks where you need to access many small files at once…

For looking up individual files, however, HFS+ is actually one of the fastest filesystems out there, and has been for a long time. This all comes from the way that HFS+ stores its data: when you’re working with relatively few files it’s better, but when you’re working with many files at once it isn’t as good. It’s a design tradeoff, and whether it will be better or worse for you in this regard really depends on how you use your computer.

The original Macintosh File System (MFS, from which HFS and then HFS+ directly descend) was created in an era when most people used floppies to store all of their data. The same is true of FAT16, which is where FAT32 comes from. Apple’s engineers decided that since floppies were so slow, people and applications would try to minimize disk access in general, and so they optimized their filesystem to work best under those conditions. It worked extraordinarily well for the time, and even today there aren’t many better filesystems for people who work under those conditions.

In other words, one of the best file systems available for the iPod is HFS+ (especially compared with FAT32). Unfortunately, FAT32 is not a comparable alternative to HFS+. FAT32’s presence as an alternative file system for the ipod simply reflects the lack of support in Windows OSes for the more advanced HFS+ file system.

Perils of FAT32 to HFS+ Conversion

As a result of learning this, I became increasingly interested in converting my FAT32 ipod to HFS+. Besides fragmentation and reliability, I also wondered if a change in ipod file systems might affect the file transfer speed over USB 2.0. File transfer speeds over USB 2.0 with my FAT32 formatted ipod averaged around 6000 kB/s. Would HFS+ perform worse or better?

General information on converting the iPod from FAT32 to HFS+ was plainly lacking and specific recommendations advised iPod users to accept FAT32. I was on my own. To access HFS+ formatted drive volumes on Windows I’d need to install special software like MacDrive by Mediafour. So to begin, I downloaded the MacDrive software and formatted my ipod to HFS+. So far so good. I wanted to make certain that my firmware was installed correctly so I proceeded to initialize my ipod with iTunes, and then re-transfer my mp3s and mp4s to the newly formatted ipod with winamp + ml_ipod. This seemed to work fine (although I didn’t see any discernible change in file transfer speeds). But afterward, I was surprised to find that my ipod was still formatted with FAT32! I soon learned that as part of its restore sequence, iTunes for PC will automatically format HFS+ formatted ipods with a FAT32 file system. It also copies ipod for PC firmware that seems tailored specifically for FAT32.

In my next attempt, I reformatted the ipod to HFS+ with Macdrive, ignored iTunes altogether, and did a full restore with ml_ipod in winamp onto the Ipod. ml_ipod recognized the drive and transferred the files. This time the file transfer speed was much higher: 9500 kB/s vs. 6000 kB/s. I was impressed but once the transfer completed, I found the ipod would not recognize any of the files that had been transferred. The itunesDB database was not corrupt and the actual data files were all present so what could be the problem? Was it a problem with the iPod’s firmware not being able to read HFS+?

I found the answer on a wiki page written for Gentoo Linux users on how to update ipod firmware. Simply formatting the ipod’s drive to HFS+ would not work because HFS+ formatted ipods have three partitions: the first partition contains the partition table, the second partition the ipod for mac firmware, and the third partition the media files and databases. (FAT32 formatted ipods have two partitions: a hidden one for the ipod for pc firmware, and the other for the media.) The ability to create these HFS+ partitions on the iPod aren’t available on Windows, even with MacDrive. MacDrive can format a disk to HFS+ but does not provide the ability to create three separate partitions on the disk. And to make the ipod work, I would also need the correct ipod firmware installed in its respective partition. Could iTunes solve the problem? iTunes for PC will neither create the three HFS+ partitions nor copy anything but ipod for PC firmware to a FAT32 partition. The only solution I could imagine for copying the correct firmware and creating the correct partitions would be by connecting my ipod to a computer running OS X and restoring my iPod using iTunes for Macintosh.

So iPod USB cable in hand, I visited my friend Isaac S. and his Macbook, and soon afterward I had a functioning ipod with the correct HFS+ partitions and firmware. (Thanks Isaac!) Back home, I found that with MacDrive installed on my Thinkpad, ml_ipod and winamp had no difficulty recognizing the HFS+ volume. Transfer speeds hovered mid 8000 kB/s. Success!

The conversion did not come without any caveats. After the full transfer was completed I did notice that there was less free space available on the ipod. The ipod with HFS+ used approximately 5% more storage for the same files than when it was formatted with FAT32. (116gb/FAT32 vs. 122gb/HFS+ out of 148gb total.) I don’t know why, but perhaps it has something to do with the extents allocated for each file in HFS+ (described above). (See update below on this weird problem.)

Because ml_ipod was designed to restore fat32 formatted ipods, I don’t think I’ll be able to use ml_ipod’s “restore or initialize ipod” feature anymore, nor will I be able to rely on iTunes for PC for the occasional firmware update. Rather than buy a whole new Apple computer for this task, I’m looking at vmware workstation, an emulation environment that I can run OS X on within Windows. Another option is to use another piece of software by Mediafour called Xplay.

Conclusion

I hope this story helps anyone else out there wondering whether to get their FAT32 ipods converted back to HFS+ (and how exactly to go about doing that). I think it’s a worthwhile project because of the advantages that HFS+ provides in speed and reliability over FAT32, the lack of file fragmentation in HFS+, and some moderate file transfer speed advantages. The disadvantages  are the need to purchase HFS+ software for Windows like MacDrive and no longer being able to depend on iTunes for firmware updates or ml_ipod for occasional full restore and ipod initialization. (You can probably get around the latter problems by installing Mac OS X in a vmware emulation, but then you’d need to buy a copy of vmware workstation and OS X as well. Or you can buy a mac mini, macbook, or other Apple computer.) If this doesn’t faze you, then you should also expect that due to differences between the two file systems, that HFS+ will utilize more storage space on your ipod than FAT32. On my ipod, HFS+ used 5% more drive space with the same files loaded onto it.

If you want to run an HFS+ formatted ipod on a PC running Windows, follow these steps:

1. If your ipod is formatted FAT32, restore it using iTunes for Mac on a friend’s Macintosh computer. (iTunes for PC will only format your ipod to FAT32.)
2. Install HFS+ reading/writing software for Windows like MacDrive by Mediafour.
3. Optional but recommended: Install ml_ipod for winamp and transfer your files to your HFS+ formatted ipod.

In the comments please let me know if you’ve found other ways to partition ipods correctly for HFS+ without using iTunes for Mac. Besides file transfer speed changes and degrees of fragmentation, I’m also interested in documenting any other reported benefits of using HFS+.

UPDATE: A week later and I’ve reloaded my ipod once more under slightly different conditions. The important difference is that this time, the strange 5% storage space loss from my earlier adventure didn’t manifest. Instead of restoring the iPod using ml_ipod, I used XPlay (ver. 3.0.2), another piece of software by Mediafour. I’m not exactly certain what made a difference… but my iPod certainly seems happier having been reformated with MacDrive and restored with XPlay. XPlay has a trial period of 30 days or 20 times running, and I’ll be curious to know whether the software makes any difference to managing an HFS+ formatted iPod besides using its restore feature. I’ll provide another update to this post when I do.

Tags: ipod
Categories: Aharonium, Ears and Geekery
No Comments »

Ten years ago I was in Philadelphia and excited to learn that Emergency Broadcast Network (or EBN for short), an art music/video project would be touring with dj Spooky providing live mixed visuals and even performing their own set. I had first seen their work in college in the mid 90s, probably on a friend’s VHS player showing a copy of Commercial Entertainment Product, their 1992 release of eleven videos on video tape. The frenetic and aggressive music on the video album didn’t really appeal to me; it was more the way they sampled video samples of explosions and machine guns firing with their audio into a coherent music (and video) collage that blew me away. Till then I hadn’t been fortunate enough to see them perform live and didn’t even realize that they were more or less an art project that had been shoehorned into the form of a touring band. (It might be a testament to how narrowly focused I was on the particular strains of ambient music that I was listening to and mixing with then as a DJ at SUNY Binghamton’s WHRS, that I missed their 1995 CD release Telecommunication Breakdown. If I had heard it I would have been amazed at the ambient stylings of the tracks “3:7:8″ and “This is the End” and I would have been enchanted to learn that both Bill Laswell and Brian Eno were involved with the release.) Shown below, “3:7:8″ :

Upon moving to Philadelphia in 2007 I bought a copy of Commercial Entertainment Product at the Digital Underground, a music store at 5th and South where I was making friends with local scenesters, and it was there that I probably learned the following year of the Spooky tour with EBN coming to the TLA. I had a mixed experience at the show. I think I got there late but was quickly impressed by EBN’s visuals. They had set up a double screen with a mirror image of the left on the right side, so there was some very cool if simple effects of action in the videos blending towards the center of the two screens. The visuals they provided for Spooky’s set were again very aggressive and I thought kind of childishly masculine, with lots of quick cut edits of men in race cars, spies, guns, and things getting blown up. EBN had made their name for videos that parodied the manipulation and dissemination of propaganda for the first Gulf War through mainstream media. For example, in their video “Syncopated Ordinance Demonstration #1″ (see below) they contrast the war footage of tanks getting bombed, with GI Joe’s cartoon battles, and scantily clad women shooting uzis in gun manufacturer advertisements, and so present the different ways violence on TV is presented in one single grotesque.

EBN’s viduals for dj Spooky’s sets were much more superficial. Without depth, EBN’s art was merely being used to complement the aggressive and masculine tone of Spooky’s presentation of illbient in relation to hip hop.

But I wasn’t dissapointed during EBN’s solo set. I saw videos that were works of art in and of themselves, and not being used to complement some other message. One of them featured a manipulation of Frank Sinatra from a short TV clip that would phase in and out of itself in audio and video. Seeing it made the entire evening worthwhile. Following the show, I searched in vain for anyone who had recorded the show. I wrote to dj Spooky asking for more information. I asked friends who new folks that regularly bootlegged shows at the TLA. Nada. And to make matters worse, I soon learned that EBN disbanded.

Fast forward to 2006. EBN videos were all over the place on youtube, and I did some exploring and found that the EBN project has been revived somewhat. All the members had gone onto other things, mostly in media production work, and EBN frontman Joshua L. Pearson had become a family man. But he had also created an official web page for EBN and posted a few videos, mostly quicktime files from Commercial Entertainment Product, for download. I still couldn’t find the Sinatra video but I was excited that it probably wasn’t lost. Hopefully it would be posted on youtube or somewhere else. At the time, looking for it would have to wait since I was terribly busy in Louisiana doing urban planning following the hurricanes of 2005. I would follow up on this later.

And so when I had some spare time earlier this year I sent out emails to all the EBN project members on whether the group had any plans to make an official release of the old videos on DVD. Greg Deocampo (currently of Mediatronica) was the only one who responded, but wow, what a response. He pointed me to his pesonal project Eclectic Method (EMN) and his portfolio of EMN videos. On a separate page of the EMN project, Greg had all the videos that had been made for the CD album Telecommunication Breakdown in 1995 but hadn’t been released due to there not being enough space on the CD for all those videos. (Only “Electronic Behavior Control System,” “3:7:8,” and “Homicidal Schizophrenic (A Lad Insane)” were released on the data side of the CD.) Mediatronica was also hosting a mirror of the videos on their video distribution site televis.es. Among the flash videos was a copy of the Sinatra video entitled “Frank”; I was overjoyed! (See “Frank” below.) A great interview of Deocampo is available in the episode archive of the public radio program, Some Assembly Required.

Having become a collector of EBN videos, I was dismayed to find that quite a few were no longer accessible on youtube or anywhere else. For years, a site called GNN (Guerilla News Network) had hosted a series of seven EBN videos it called “The Lost Tapes.” A few had surfaced on youtube, and one or two on file sharing networks, but the others had since 2004 when GNN stopped hosting them, become truly lost. Another video, “Banjo Lesson,” was made inaccessible when a youtube user named Nomeus had his account suspended. And so last week, I went looking for Nomeus, and finally caught up with him on his urban exploration site flurbex.com. I’ve since been able to get copies of all the missing files and repost them on youtube. Here’s “Banjo Lesson”:

Tags: collage,collecting,EBN,video
Categories: Aharonium, Ears, Film, Mythology and Poetry
No Comments »

An introduction to Piyutim (piyut.org.il)

A piyut (piyutim, pl. hebrew) is a sacred musical poem, sung as part of a communal prayer service but just as often after a good meal with friends and family. I was raised with these songs and tunes, learning a new one occasionally while eating as a guest at someone’s house, or at a weekend gathering, or in Israel at Yeshiva. I always hoped there was some archive because I was hearing quite a few of the common melodies and worried that there were likely thousands more that were fading into obscurity or limited by geography. (Ever wonder what shabbat tunes are kept in the piyutim of Kurdish Jews?) Then I stumbled on this site, piyut.org, which is just such an archive. I am so thankful. They even have something like a comprehensive collection of musical scales… I’m not certain what is meant by “musical scales” on this page, but I chose one at random and I found some musical expression that was completely new to me. I suspect that the music found on this site would also be appreciated by audionauts of sufi sacred music such as the Qawallis of Nusrat Fateh Ali Khan. But this archive is so diverse, I am still plumbing its depths of ancient sounds and their contemporary echoes.

I don’t know when this website was founded but from their about page it seems quite active with a passionate group of musicians, academics, and other scholars working on something they know is unique and essential to preserve and promote. This statement on that page summed it up nicely:

The vast majority of the poetic and musical creativity of the Jews emerged in various Diaspora communities during the past two millennia. Since the founding of the State of Israel and the immigration of the majority of these ancient Diaspora communities to Israel, large sections of the great tradition of piyut have been lost or forgotten. Finding access to the remnants that remain is not easy. The brief history of the modern period created, in many cases, a gap between the tradition of the past and the modern society and culture that developed in Israel. Tradition generally, and the legacy of piyut in particular, has stayed alive and meaningful only among a small portion of the Israeli population.
As time has passed, the need for people to connect with these roots has grown greatly. It is a need to access the voices calling from the depths of time, absorbed in emotion and wisdom of the many generations that sang these piyutim. We will widen and deepen our language and understand ourselves and our nation better as part of understanding our ancestors and their traditions better.

Categories: Ears and Yiddishkeit
No Comments »

This is an informational post for ipod classic owners out there. The recent firmware update 1.0.3 changed the functionality of the shuffle songs feature. Until you follow the following steps, the menu setting for “Shuffle” will have no effect.

To change the ipod from shuffling songs to shuffling albums follow these steps:

1) Go ahead and shuffle your songs
2) Press the center button three times
3) select the shuffle setting you want (”songs”, “albums”, or “off”)

Once you have done this you will once again be able to change the shuffle setting via the ipod classic Settings.

For regular readers of this blog curious whatever happened to my 100gb Archos Jukebox/Rockbox player (blogged of here) a short note. The Archos is ok but (since soon after the periodic completion of this spaceling’s ambient solar orbit) the Archos took to rocking it with the Junkions on Planet Junk (at the bottom of my junk pile).

But looking back all nostalgia like for my tech-Gomi, the Archos Jukebox was an energy whore, that required me to lug around an AC adapter to service it’s unquenchable thirst for electrons. And it couldn’t shuffle albums.

“Shuffle Albums” was the killer feature I pined for in an mp3 player so when I discovered the ipods were capable of that (and the newer archos’ and cowons still could not) I shelled out for my little slice of Apple. (iTunes free thanks to Ubuntu Linux + Floola, I should add).

Why is shuffle albums so important? For the same reason that “Album List 2″ is an essential plugin for winamp — it provides for the ability to listen to a random album (just like those old giant 100 CD jukeboxes could). Industry folks talk about how the revolution of the mp3 was that it liberated tracks from the medium of “Albums.” Tracks could be listened to individually, mixed and shuffled at random, and most importantly, sold individually. But the medium of albums is still important for much of the music I listen to, whether it be Beethoven’s 9th Symphony or Steve Reich’s Music for 18 Musicians: I want to listen to my tracks sequentially in the order suggested by the composer artist’s imagination. Or said differently: context matters, provenance matters, and please don’t use tech to manipulate art as a commodity.

The Archos is still accessible and usable and I’ll find a place for it in some future project. But aside from random firmware scares (see above shuffle surprise), I’m pretty damn pleased with this little apple unit. Wish Apple’d be more open with their firmware though and less jerky about opening up their tech to 3rd party itunes alternative ipod managers.

Categories: Ears
No Comments »

Pink Floyd fans may know of this series of discs that make available a plethora of rare recordings: radio adverts, interviews, mono edits, alternate versions, etcetera. On disc three, track 20, a very special track, and one which gave me the shivers. You’ll understand when you listen to it.

From wikipedia:

Snippets of dialogue between and over the top of the songs are also featured on the recording [of Dark Side of the Moon]. Roger Waters devised a method of interviewing people, whereby questions were printed on flashcards in sequential order and the subject’s responses were recorded uninterrupted. The questions related to central themes of the album such as madness, violence, and death. Participants were commandeered from around Abbey Road, placed in the darkened studio in front of a microphone, and told to answer the questions in the order which they were presented. This provoked some surprising responses to subsequent questions. For example, the question “When was the last time you were violent?” was immediately followed by “Were you in the right?” (Henderson, Peter; Surcliffe, Phil; and Bungey, John (1998). The First Men on the Moon Part 2 (html). REG . MOJO Magazine. Retrieved on 2006-12-23.)

Recordings of road manager Roger “The Hat” Manifold were the only ones obtained through a conventional sit-down interview because the band members could not find him at the time and his responses (including “give ‘em a quick, short, sharp shock…” and “live for today, gone tomorrow, that’s me…”) had to be taped later when the flashcards had been lost.

From the liner notes of A Tree Full of Secrets (disc 3):

Roger the Hat interview (07:30)
Recorded : 1972
Primary source : Capital Radio broadcast (”Pink Floyd Story”), December 1976 - January 1977
Source for the tree : unknown gen from Capital Radio broadcastAll through “The Dark Side of the Moon” album, you can hear the voices of people interviewed by the band. Here, Roger the Hat (who was a roadie for various rock bands) is interviewed by Roger Waters. On the album, you can hear his voice during “On the Run” (”Live for today, gone tomorrow, that’s me, hahaha…”) and during “Us And Them” (”Short, sharp shock …”).

(UPDATE: Very difficult to find more information on Roger Manifold on the web. The above picture came from this webpage about another band he was a roadie with, The Third World War. Apparently, the Hat monicker derived from Mr. Monicker’s recognizable top hat, sported in his daily roadie ministrations. Wish I could find a picture of it. I love top hats, too.)

Categories: Ears
No Comments »