End of an Era

Sam Hart

2007-05-02 14:28:52

For those who haven't seen the tiny coverage it's getting, Progeny Linux Systems has shut down. From the website:

We are sorry to inform you that Progeny Linux Systems, Inc. ceased operations April 30, 2007. Questions may be directed to:

board_at_p.c

Offers to purchase the progeny.com, progenylinux.com, progenylinux.org, progenylinux.net, componentizedlinux.com, componentizedlinux.net, componentizedlinux.org, or globalinuxsupport.com domain names should be made to:

webmaster_at_p.c

As I mentioned here, I recently left Progeny to work for the Linux Foundation (which, itself, was the child of an Open Source Developer Labs and Free Standards Group merger). As I said in that post:

Naturally, there's more to the story than that, and maybe someday I'll relate it, but not now.

Well, I'm still not 100% comfortable relating the whole story. However, I would like to comment on some things.

Progeny: The distro company

Astute readers of my site may already know that I originally made the switch to a Debian-based distro through Progeny's original distro back in 2001. You can see my orginal screenshots from that occasion, as well as read my entry on the subject.

Progeny's PGI installer was, for the time, the most impressive installer out there. It came out a good year or so before the graphical version of Anaconda. It was well before graphical installers were seen anywhere else (even including Caldera's famously "easy to use" installer). As someone who genuinely knew the hardware he was installing on, and who was used to having to hold the hand of every installer as I installed new distros on my systems, I was amazed at how easily PGI picked up what hardware I had and pre-configured it for me. In fact, I'd say PGI spoiled me for years to come, and I actually kept using that ancient Progeny CD to install Debian bases for years to come (you just had to know how to upgrade to modern Debian properly :-)

What a lot of people don't know (which is so evident when you read the comments on the OSNews article) is that Progeny left the distro market pretty quickly. In fact, by the time I was using Progeny Linux to install Debian, they may have already been out of the market. I can't really blame these people, I guess. When I went to work for Progeny in 2004 I actually still thought they were in the distro market.

At any rate, PGI, as cool as it was, floundered and is now gone. Perhaps I should check to see if the SVN is still up and grab the PGI source before it vanishes forever.....

Progeny: Custom distros and Transition services

I'm pretty sure Progeny did other stuff to survive until I showed up, but when I showed up there were really two bits to Progeny: The custom distro shop and the transition services shop. I worked in the custom distro part, so I wont dwell much on transition services except to say what it was and the roll it played.

Progeny Transition Services, PTS

PTS was a pretty good idea. There's all these businesses and organizations out there who, at one point, set up some massive Linux-based something. Could be they are a grocery store chain with a bunch of Linux-based teller machines. Or maybe a national bank that has Linux-based servers in every branch. Doesn't matter. All that matters is that at one point they spent a large amount of money setting up and rolling out some Linux-based something.

Well, when the roll-outs are costly enough, and the persons who set them up not knowledgeable or experienced enough to make them easy to upkeep, you find there is a tendency to let this technology slide. Years pass, and no updates are patched to the systems. The software gets old and decrepit, becomes unstable and insecure, and the companies/organizations start desperately looking for options.

This was where PTS came in.

PTS would help these companies transition their old and sickly Linux-based systems to something modern. PTS would backport security and bug fixes to these ancient systems, as well as sometimes try to help these companies migrate themselves out of the corner they had painted themselves into.

Progeny Platform Services, PS

This was the team I was a part of. PS was the custom distro team.

There's all these needs for customized distros, ones that are built for a very specific use. Let's say you're designing a Linux-based cell-phone, well you can't very well take RHEL and cram that sucker into your tiny embedded environment, can you? Or maybe you're building some locked down boxes that route cell phone calls and that will be operating in some shed in the middle of the Sahara or the Antarctic where they wont see a maintenance person for years, can you really be certain that CentOS is going to give you that kind of uptime and stability?

PS was where customers with needs just like those would come. We would build custom Linux-based solutions to the most wild and hairy needs. Our software really was everywhere. It powered the Nokia 770, powered various other network appliances. Working on PS was always exciting and new, because nearly every customer had different needs.

Progeny Componentized Linux

CL was slightly a radical idea. The concept was, what if we had a Linux distro that was modular, whose pieces could be used to build other distros from? Think a Lego set for Linux. Sure would make our jobs in PS easier, no?

There were two aspects of CL: The Debian-based one and the RPM-based one. The Debian-based one was mainly headed up by others, but I was in charge of the RPM-based one. It was called CLRPM, and was based on CentOS. It had a self-contained little core (ready for to be embedded, if needed), and had modules that could be added for other various distro needs. It had this fancy customizable Anaconda-based installer (with a simple command, you could rebrand the whole thing)

However, there were two big problems with CL.

Number one, the Debian-based CL was redundant, IMHO. Debian is already highly modular. It's full of meta-distributions, already has interchangeable pieces, and has tools for building custom Debian-based distros. All you have to do is look at the myriad of Debian-based distros out there for proof. Ubuntu, Knoppix, etc. wouldn't exist if it weren't for the already modular nature of Debian.

Number two, the CentOS-based CL was never what our customers wanted at the time. The problem Progeny had was we were always playing catch-up to what our next and new customers wanted.

We'd have one customer that absolutely needed a customizable and brand-able Anaconda installer. So we'd spend weeks getting it just right for them. We'd perfect the whole rebranding Anaconda thang. But then we'd never get another customer with those needs again.

Or maybe we'd have a customer who needed RPM-Autorollback (The ability for RPM to automagickally roll back transactions in the event of some problem so a system will, theoretically, never go down). So we'd spend weeks getting this code perfect, coordinating with upstream sources to get it mainlined, and then we'd never get another customer who wanted it again.

Or we'd have another customer who needed gensplash wedged into a stock Fedora Core kernel. So we'd spend weeks making it fit and work well, only to never have another customer needing it.

The end result was that CLRPM never really was what our new customers wanted. It was always what would have made our previous customers thrilled to have, but the features were always lagging behind what our actual paying customers wanted.

Whither Progeny

I think the problems for Progeny started before I got there in 2004, but they really did come to a head in 2005. Sometime in 2005 they shut down PTS, which accounted for a large chunk of the company's revenue. PTS was being shut down because it was always intended to be a "transition" service, and the idea was that people should be transitioned by then. Unfortunately, they weren't, but we shut down PTS anyway. There were two or three PTS customers who still paid us to perform PTS updates for them (in fact, up until Progeny finally closed its doors, we were still referring to those customers as PTS customers even though PTS was supposedly gone), but PTS on the whole was gone.

Additionally, we had a slew of nasty PS customers. Somehow our contracts never quite protected us from customers walking all over us, and we kind of got screwed a lot.

In 2005, we had a layoff session and went down to 8 employees. I was one of the 8 that remained, but in the next year and a half of employment I would sometimes wish I hadn't. Or problems really only got worse, and the job became fairly hard.

We managed to get in some more nasty contracts, including one where a customer wanted to have us work for a month "for free" to evaluate whether or not they wanted to keep us. The final contract that we were working on before I left was undoubtedly the most painful. It honestly shouldn't have been, as it was some interesting kernel/userspace work that otherwise would have appealed to me. But the customer was just such a pain to work with, and would argue and quibble over every payment. The end result was we'd actually go without pay for a while due to this customer throwing some tantrum. All of us were pretty well burned out by this point, and this childish customer wasn't making things any easier.

I had been working contracts for the LF since Dec. 2006, and when I was offered a job there I kind of leapt at it. I hate to say it was like a rat fleeing a sinking ship, but less than a month later Progeny was gone.

At any rate, in spite of some of the pain that I experienced while working there, I really do value the time I spent at Progeny. I made some very good friendships there, and I enjoyed most of what I worked on. Honestly, if the revenue had been more steady and they didn't keep getting contracts that allowed customers to walk all over us, I would still be there. The job was enjoyable, just overly stressful.

Anyway, I've rambled enough. I wish my former coworkers well, and encourage anyone who may be looking for some talented Linux-persons to contact me at



and I'll get you in touch with talented (yet unemployed) former Progeny employees.