YetAnotherPersonalHomepage

My brother has compiled a fairly extensive genealogy chart. As it happens one of our relatives is in New York vacationing and Joe is in Michigan, so I agreed to meet him for lunch. I did so mostly as a favor to my brother but also with a little curiosity- I never met more than one or two of my maternal grandmother’s relatives and thought it would be interesting. To trace the connection, Dan’s is of my mother’s generation and his grandfather, Josef, was brother to my great-grandmother, Regina, for whom I am named.

Dan and his companion Elisa, both recently retired and visiting from Isreal, met me in my company’s lobby and he threw a hug around me, something of a surprise for a first meeting. We spent the next two hours chatting over lunch at My Favorite Dessert, a kosher New American-style restaurant (didn’t this used to be called ‘bistro’?). I have to note that the place, recommended to me by a coworker, has very good food and pretty prices reasonable for a mid-town restaurant, but it was terribly noisy- packed with families and tourists, many of them Israeli (Elisa saw someone she recognized from home). It was like meeting a friend you had not seen in ten years, very comfortable and nothing uneasy about it. Dan had previously talked with Joe about the family tree so he didn’t have much new material for the chart, but I did take their picture and snap a shot of a collection of old photographs of his father, Walter. He also gave me copies of his father’s birth certificate, passport and immigration papers. Cool stuff. Dan said he would forward copies of others and other photos he had at home to Joe.

I’m going to see if Joe will let me put his data online along with copies of documents. This is very interesting stuff and I was great to meet members of the extended family.

What looked like a easy task- knock together a CGI in perl to query a db and generate a graph- has turned into a real mire. I have spent several days fighting with doing DHTML on NS4.x on Unix, NS4.7 for Windows, IE5, Mozilla/NS6, Konqueror and IE6. Paraphrasing a line from the text adventure “Zork”: You are in a maze of twisty little standards, all different. In the past I’ve avoided doing complicated user interfaces, leaving for people who know what they’re doing to fight and refight the browser wars. It mostly works on Unix, still working on the IE stuff. At least when I’m done, I’ll have a nice little store of knowledge and a library of useful functions.

Anyway, I got thinking about what was in the box itself. I’ve seen it reboot and noticed that it has both warm and cold. I’ve occasionally seen it screw up, dropping frames or audio and even lock up. The form factor suggested that it was built around embedded commodity chips with a MIPS or StrongARM CPU.

I’ve met cable installers and figured that if I were designing such a beast I would include a simple operator interface accessible from the screen. I spent some time punching in unlisted channels (it seemed likely that there were ones not on the interactive guide) and found that there are two #1999 and #996 which are a control panel and a system summary (I also found that there is a silent video feed from what looks like an apartment building lobby surveilance camera). It seems that the box has both analog and digital connections and runs on an private IP network in the 10.0.0.0/8 range. The MAC for the box is from the range assigned to Scientific-Atlanta Arcodan of Denmark (00:01:a6), which isn’t particularly useful given that the box is clearly marked as theirs. Poking around I found it has 7.5MB RAM free and about 4MB taken by a framebuffer. The diagnostics register the presence of a full collection of ports: serial (probably the IR), scsi, usb, ethernet and a card slot. With the exception of the card slot and usb, these ports are not externally exposed. It keeps basic stats about uptime, retries, failures and some very limited data on purchases but a fairly large amount on when and how many ads are seen on the PPV and MOD channels.

A little searching around the internet turned up documentation from the manufacturer. The Scientific Atlanta Explorer 2100 box is based on a 130MHz RISC (turns out it’s MIPS) with a board that comes equipped with 16MB of RAM- 8MB RAM, 4MB graphics RAM, 4MB EEPROM, and MPEG2 decoding. My guesses about what was inside were pretty good. I’d love to get my hands on a broken unit and take it apart.

New York City is probably the largest and most lucrative single consumer market but that didn’t keep Time-Warner from blowing a lot of customer satisfaction on Friday night by hosing the Movie On Demand (MOD) feature of their digital television service. In our case, after we had purchased a movie and watched forty-five minutes of it. Calling their customer service number gets you to the usual shallow menu of numbered options where any regarding service delivers you to a recording that there is an outage for, essentially, all of Manhattan and the Bronx above 42nd street and most of the East side. No explanation and no time frame for restoring service. Nice job.

Time-Warner, if they are looking far enough ahead, know that Movies on Demand is just one of the dishes on a menu of pay per view and on-demand services on top of the monthly service fees. Missed an episode of Buffy? Watch it when you want for $1. Can’t stay up for your mysteries on BBC America? Time shift it for a $1. Don’t subscribe to HBO but want to see the Sopranos or Sex in the City? Sure, for $4 it’s yours. Want to see all of last seasons episodes? No problem, $40 and you have a month to watch it. What ever was broadcast you can see it at your convenience. It sounds like the pre-dotbomb Qwest commercials. Instead of feeling nickled and dimed, people will pay. They’ll pay because it’s priced just about right for the entertainment they are getting and it’s most of the convenience of Tivo without buying another box- the personal video recorder (PVR)- or another monthly subscription. Best of all you don’t have to leave your couch.

What that cable operator’s utopia depends upon is:

It would seem, from this episode and from horror stories on popular boards like dslreports and Slashdot, that what the cable companies know about designing stable, highly available IP-based networks would fill a thimble. Without predictable reliable service available when and where the demand exists, they are not going to have happy customers. People want what amounts to “dialtone” from the cable company- it’s always there and you can assume it’s always there. There’s no technical or physical reason that the cable companies can’t repeat the Bells accomplishment except that doing it is costly and they either don’t value the new business enough to get it or are hoping that they can slowly grow into it, paying off the necessary investment as the market develops. That means they’ve already ditched three and four, so they’re going to have to make it on the first and second to prop up the last. And that is going to be a hard row to hoe.

Unfortunately, it looks like Time-Warner isn’t getting it. Their pricing is reasonable but not exactly cheap- $4/movie, about the same as a rental- but the value leaves something to be desired. The number of movies is limited. This is probably imposed by cost, namely storage of the MPEG2 files each of which are likely in the 500-1600MB range. I estimate this size because it is unlikely that the movies are encoded at a resolution greater than that supported by the lowest set-top decoder, 720×480 (maybe even down to 480×480) with 65k colors and from my viewing I suspect they use a frame rate of 15-20fps. Another problem with the value proposition is what is presented was mostly not worth seeing in their first or second runs. That makes an impulse buy over take-out on Friday night less likely. The static, somewhat longer than monthly holding of the choices turns people away when they go there looking for something new. This is little more than a reformulation of pay-per-view when people really want a PVR.

The patent held by Unisys on the Lempel-Ziv-Welch (LZW) compression scheme expires tomorrow June 20th, 2003. This is a good thing since the compression is used in the gif image format and the Unix compress among other things.

Unisys probably deserves its place as the first whipping boy for software patents. Set aside the fact that an algorithm is not in and of itself an implementation so it should not be patentable as a machine or the matter that by granting such patents the Patent Office paves the way for broad and speculative land rush tactics (think Amazon, among others). Unisys irked a lot of people with what looked like a bald faced money grab.

A little background: Lempel and Ziv worked at Sperry and while there were issued a patent for the LZ78 scheme. Welch had worked for Sperry and published his enhancements after leaving them but Sperry was issued a patent for the work. Sperry merged with Burroughs to form Unisys in 1986. You can pull all this from some well crafted googling and the Unisys website.

What happened next is open to dispute. In 1987, Compuserve introduced the gif format which made use of the LZW compression scheme. Either Compuserve didn’t do its homework or Unisys turned a blind eye but the result was that gif came into common use and Unisys did not enforce its patent regarding gif until 1993. That’s when people got angry- the world wide web was taking off and most images were in gif format, a format they assumed was free to use, but here was Unisys claiming that they were due for every encoder and decoder written. An interesting footnote, IBM also holds patents on compression and, it’s reputed, on earlier and later work that invalidates or overlaps those owned by Unisys. IBM seemed to choose to ignore the small fry. In any event, the timing couldn’t have been worse and Unisys got a well-deserved black eye among technical people.

The above linked patent, if you tried to read through it, describes an implementation in hardware. The algorithm, first published in 1977, can be pretty easily explained:

    Starting at the beginning of the data, look for unique strings. If
    the string is new, write it to a table. Replace the string with a
    single character code representing the index to the table for that
    sequence in the output. Continue through the data matching
    strings, adding news ones to the table and outputing codes.

Getting even farther afield… it naturally isn’t quite that simple in practice. For one thing, you want to only replace strings of more than two bytes in size- otherwise there wouldn’t be any savings- making the algorithm trivially more complicated. Another limitation is that the table is going to be of limited size- there are only so many unique strings you can recognize so a largely random file is going to blow your table and ruin the resulting compression. One more is that you need to reserve a particular bit pattern as a flag that the byte(s) following it represent an entry in the table, so for some sequences you will end up expanding single bytes into multiples. And you’ll need to reserve a bit pattern to represent the end of the file. It’s still reasonable to implement and if you do like a lot of implementors, you use it with Huffman encoding or Run Length Encoding you can squeeze the data even further.