www.nickhodge.com

“Stall­man: Linux used to track Lon­don­ers”

Finally, Stall­man sug­ges­ted keep­ing Oyster cards in alu­minium foil when they aren’t actu­ally being scanned for travel, to pre­vent them being scanned secretly.

Watch the video here of Frank Arrigo and Monique Eagles here. Yes, you will need to install Silverlight.

This is my first exper­i­ment with Sil­ver­light and the Microsoft Expres­sion set of tools. Using the inbuilt play­ers in Media Encoder saved many days/hours of hand cod­ing; yet I am sure there is more in there that will tickle out over com­ing weeks.

NOTE: Sil­ver­light 1.1 is alpha-release!

Work­flow (all on Vista Ultimate):

• Edited foot­age in Adobe Premiere Pro 2.0
• Export Sequence from Premiere Pro using Adobe Media Encoder 960×720 WMV9/WMA9, very light compression.
• Import into Microsoft Expres­sion Media Encoder (May preview)
• Export foot­age as VC-1 Web Server High Speed (using a nor­mal web server). This set­ting is 640×480. Obvi­ously, I could com­press this more.
• Edit the Default.html to cor­rectly ref­er­ence EmePlayer.js (note: this got me for an hour. Linux web serv­ers are case-sensitive, and the Default.html points to emeplayer.js. 404! Bug reported)
• FTP files to dir­ect­ory onto nickhodge.com (could have used Expres­sion Web, but I was debug­ging the prob­lem with upper/lower case file nam­ing above)

Thoughts? Com­ments?  I only have Sil­ver­light 1.1 alpha installed. I’ve tested in Win­dows IE/FireFox and MacOS X 10.4 Safari/Firefox. The Mac’s audio might be out-of-sync. Again, this is reported.

Spent the last week and this week on a per­sonal Ruby on Rails pro­ject. This involves sub­ver­sion (as a ver­sion man­age­ment sys­tem), mon­grel, cap­istrano, ftp, post­gr­esql, some smarts with DNS, exim and a two-day com­plete re-install of Debian. That re-install was not expected.

Unix has this won­der­ful and power­ful concept: the root user knows what they are doing at all times. 99.9% of the time this is a safe assump­tion. 0.1% of the time you type “yes” instead of “no” — remov­ing the ker­nel in this fash­ion is highly not recommended.

How do you fix a broken Linux install?

Stage one involved mak­ing what is known as a LiveCD, or boot­able Linux. I decided to down­load and boot from a Knop­pix LiveCD. A quick restart from the CD, and I could see that the data was intact.

Stage two was installing a new 350GB HD for the data to bring the server up to 0.5TB of stor­age. The old faith­ful Unix stand­bys of dd, fsck from the old 200GB to the new 350GB and start the dif­fi­cult work.

Stage three is a full Debian rein­stall onto the old 200GB with 0.5Gb down­load of the most cur­rent pack­ages, and re-apt on a 686 rather than 386 ker­nel. This didn’t take too long. Re-configuring all the serv­ers and ser­vices: dns, dhcp, CUPS, Samba, Apache, sub­ver­sion, rails+gems, python took most of the weekend.

Stage four: backup scripts. 0.5TB is too much of a moun­tain of data to lose.

Just over the digital hori­zon, your Apple Mac­Book will boot mul­tiple MacOS X 10.x, Win­dows Vista and Linux/Ubuntu oper­at­ing sys­tems at launch. You don’t see all of their friendly faces, but they are there ready-to-go. Get-info on Fire­fox, and you inform the primary OS which of these oper­at­ing sys­tem envir­on­ments you would like the applic­a­tion to launch into. Need to run Out­look 2007 for your large organ­isa­tion? No stress: its there, behind the scenes as you run­ning Pho­toshop on the MacOS side. If your Win­dows instance crashes as you are test­ing a new applic­a­tion; it is shut­down and relaunched sim­ul­tan­eously. Far fetched? With Vir­tu­al­iz­a­tion and mul­tiple Core CPUs, no. On the desktop, usable vir­tu­al­iz­a­tion is rel­at­ively new; what can we do with mod­ern pro­cessors and soft­ware. There are cycles to burn.

Vir­tu­al­iz­a­tion is noth­ing new in the IT world. Main­frames, the Cro­codilian sur­viv­ors of last cen­tury have long used Vir­tu­al­iz­a­tion as a method of per­form­ance man­age­ment and isol­a­tion. If you are login as user on a main­frame in, your ses­sion is a vir­tu­al­ized instance of the whole oper­at­ing sys­tem. This isol­ates you from other users, pro­tects the whole sys­tem from strange things you may do: like run­away quer­ies. SELECT * FROM INVOICES;

Micro-processors since the Intel 4004 have sur­vived with a single Cent­ral Pro­cessing Unit (CPU) that stepped through the com­mands in strict mil­it­ary time as soun­ded by the clock. We star­ted with Hertz, then x1000 to Mega­hertz and now x1000 again to Giga­hertz. A big leap has been the mul­tiple “cores”, or sep­ar­ate CPUs added to the Intel pro­cessor line, with the VT-x,With multi-core pro­cessors being the cur­rent “thing”, and with Intel talk­ing about their pro­cessors hav­ing upto 32 cores in 2010 — per­form­ance is going hori­zontal (more CPU cores) and ver­tical (Ghz Clock speeds). The drum­mer in that mil­it­ary band must be get­ting sore arms beat­ing that fast!

PowerPC and other RISC-like pro­cessors star­ted to split their pro­cessing com­mands to internal co-processors. This is like del­eg­at­ing the “dif­fi­cult” jobs to the spe­cial­ized under­lings. For instance, Maths to Float­ing Point Units (FPUs). To gain per­form­ance dur­ing the lat­ter days of the PowerPC, Apple star­ted adding mul­tiple pro­cessors to split the work­load. Smart soft­ware could take advant­age of these to speed up heavy pro­cessing tasks.

Mul­tiple Cores in the same pro­cessor is bring­ing this same philo­sophy to every­one — includ­ing low-end Mac­Book laptops. Few applic­a­tions scale well into multi-cores: one that does is Virtualization.

How does this related to Vir­tu­al­iz­a­tion? Now we have mul­tiple cores, one poten­tial use is to run sep­ar­ate oper­at­ing sys­tems on these mul­tiple core CPUs; and each oper­at­ing sys­tem run­ning at full speed. Vet­er­ans in the Mac world would remem­ber the first ver­sions of non-Microsoft Vir­tu­alPC. This soft­ware emu­lated (and dynam­ic­ally recom­piled!) the Intel pro­cessor com­mands on PowerPC like a human lan­guage trans­lator. Trans­lat­ing takes time, and there­fore the oper­at­ing sys­tem just didn’t feel snappy. Now on we have Intel mul­tiple cores, the OS world is a user’s oyster as the trans­la­tion is no longer required.

As Vir­tu­al­iz­a­tion is in the hard­ware, the soft­ware that has been unique can be com­mod­it­ised. This has drawn WMWare and Microsoft into a war for the “Vir­tual Server Plat­form”, and the first battle on the desktop to grab the hearts-and-minds. The dol­lar returns for these organ­iz­a­tions lies with the “own­er­ship” of the server platform.

Microsoft has released Vir­tu­alPC 2004 as now a free product, with Vir­tu­alPC 2007 also fol­low­ing for free. This is a parry to the VMWare thrust of WMWare Player. Vir­tu­alPC comes from a product star­ted by Con­nectix. Announce­ment made at a recent Mac­World that Vir­tual PC Mac would sup­port Intel based Macs in the future. The Channel9 Team at Microsoft have an inter­view with the engin­eers of Vir­tu­alPC. This provides an excel­lent back­grounder on the his­tory, and tech­no­logy of virtualization.

EMC, and spe­cific­ally their VMWare divi­sion, has free ver­sions of VMWare Server and VMWare Player avail­able for free. Con­veni­ently, VMWare have links to pre-created VMWare images, or “appli­ances” as they name them.

Server Appli­ances: one of the VMWare images avail­able is a Fire­fox Browser appli­ance.. A self-contained oper­at­ing sys­tem with user inter­face (Ubuntu) and Fire­fox — pre-installed, ready to go. We are all wait­ing for VMWare for the Intel-based Macs.

The recent scrabble over the Xen envir­on­ment for vir­tu­al­iz­a­tion between Microsoft and VMWare, and recent com­ments by Nov­ell are an indic­ator of the import­ance of server vir­tu­al­iz­a­tion. Many large server hard­ware com­pan­ies must con­cerned about the impact on their busi­ness as server hard­ware is consolidated.

Server, large sys­tems, IT shops with many phys­ical serv­ers and the need to sup­port leg­acy server applic­a­tions bene­fit from Vir­tu­al­iz­a­tion today. How about desktop environments?

One of the first applic­a­tions I installed on my Mac­Book Pro was Par­al­lels Desktop for the Mac. There are a few Windows-only applic­a­tions I need to use; and within 30 minutes I was sold (and yes, I pur­chased a copy). I use Win­dows XP daily, Par­al­lels has been use­ful in see­ing what this com­mo­tion about Ubuntu has been about, and doing a test install of Debian prior to pro­duc­tion install as our home server.

10 Ways Vir­tu­al­iz­a­tion Will Change Our Digital Lives:

1. Crit­ical Applic­a­tion isol­a­tion In our daily computer-lives, there are applic­a­tions that are more miss­ing crit­ical. Ima­gine you have a soft-phone run­ning via VoIP. You really want that applic­a­tion run­ning, rain hail or crash. With Vir­tu­al­iz­a­tion, you could put that applic­a­tion in an envir­on­ment that is isol­ated from your World-of-Warcraft envir­on­ment. In fact, you have a “play”, “work” envir­on­ment to sep­ar­ate your digital life.
2. Hol­istic Backups Backup your VM, and there­fore your whole envir­on­ment. I remem­ber the first time I lost a hard drive (an amaz­ing 100Mb external SCSI drive on a Mac II) and lost all the data. Bour­bon and Coke soothed the loss; now I have a laptop with 100Gb. Learn­ing from the ori­ginal loss of data, there are a myriad of backups lying around: burnt CDs, DVDs, Debian server with 450Gb, some data inside GMail. How­ever, the invest­ment in the setup of my machine is worth 4–5 work­ing days to become “online” if I lost the hard drive. With an image of the think invest­ment in the setup, let alone the data.
3. Pre-setup Server Plat­forms:A pro­ject like Ruby on Rails. To install this beastie, you have to install Ruby, Apache, MySQL (or Post­gres), the Ruby Gem remote installation/dependency sys­tem, Rails plus a few Ruby hooks to con­nect all this together. For the faint of heart, or non-OS tweaker — its a major and poten­tially impossible chore. Deploy­ing this in a development/test/production cycle increases the com­plex­ity by an order of mag­nitude. Serv­ers, and server soft­ware is just too darn com­plex for the aver­age developer.

   Instead, if there was a light­weight OS with these pre-configured, installed, secur­ity checked — a developer could simply down­load the VirtualPC/VMWare image and start.

   It would be cool if organ­iz­a­tions that sold/developed server tools provided a “pre-installed” image rather than an a myriad of inter-linked installers and required dependencies.

4. Light­weight Server oper­at­ing sys­tems, or based “frame­work” on which “vir­tu­al­ized” sys­tems can be cre­ated and eas­ily deployed. Guess what? there are these envir­on­ments for Linux and poten­tially for Win­dows.

   Rather than build­ing a large, com­plex OS with mul­tiple soft­ware sub-installs, com­mer­cial server applic­a­tions should move to a VMWare/VirtualPC image for double-click launch-and-go.

5. Deep debug­ging: ima­gine an instru­mented, or as Microsoft call it an enlightened OS. Run the VM, Record all the com­mands until crash. Rewind, Replay.
6. Vir­tu­al­iz­able OS for Test/Development; or Quick OS Undo: launch a VM and test an applic­a­tion and keep it sep­ar­ate from your work­ing oper­at­ing sys­tem. If some­thing “breaks”, throw away the VM image and restart. Keep­ing clean images of vari­ous oper­at­ing sys­tems in “cold stor­age” per­mits quick reset­ting back to a baseline.

   In a recent pod­cast Ruby on Rails inter­view with : “Its about breadth, not depth”; as pro­cessors are start­ing to go mul­tiple core, applic­a­tions need to scale hori­zont­ally, not expect more pro­cessor speed to magic­ally work. This has an impact on Dynamic/Scripted lan­guages being deployed as web applic­a­tions. Also SOA is about these applic­a­tions con­nect­ing to each other between dif­fer­ent servers.

7. Hos­ted Serv­ers from Vir­tu­al­ized Images: grab a “copy” of a vir­tual server from your new hos­itng pro­vider; loc­ally test/install & and then remotely deploy and start. After writ­ing this, I notived that TextDrive is offer­ing Sol­aris Con­tainer style serving.
8. Clone + Clone back: Take an image of your cur­rent VM OS con­fig­ur­a­tion. Make some change (install an applic­a­tion or OS upgrade), and bin­ary re-merge the now mod­i­fied clone back onto the ori­ginal. This is prob­ably more dif­fi­cult to do than think through, but a well archi­tec­ted OS could per­mit smal­ler diff style changes.
9. Why re-invent the wheel? Large applic­a­tions have large prob­lems; sav­ing files. found­a­tion: with IO/MM and low-level access to the hard­ware, could a large applic­a­tion run in its own oper­at­ing sys­tem? Sep­ar­at­ing the IO/MM out of the large frame­work into a well-tested OS res­ults in re-factored frame­works. Higher level applic­a­tions really don’t know they are in a vir­tual environment.
10. Net­work Vir­tual desktops. A twist on the Cit­rix envir­on­ment, but the CPU in use is on the desktop, file image on the server. VMware is a part of a gorup work­ing on desktop VDI. What we need is a small, eas­ily Vir­tu­al­iz­able (and pre-installed image) MacOS X and Win­dows XP/Vista.

Vir­tu­al­iz­a­tion, as the pro­cessors go madly hori­zontal with mul­tiple CPU cores, is going to be a large part of our future: not only on the server, but also on the desktop.

Down­load Par­al­lels (if you on an Intel Mac) or VMWare Player / Microsoft Vir­tu­alPC and give it a whirl and exper­i­ence the begin­ning of our vir­tual future. Burn those Cycles!

Bed­time Reading

• An excel­lent roadmap of vendors and products in the mar­ket­place, and a roadmap for the future from selec­ted vendors.
• How to Install Win­dowsXP with Par­al­lels.
• Vir­tu­ally Vista.
• Peter Nor­vig on com­plex­ity of server setup We deal with mil­lions of Web mas­ters who can’t con­fig­ure a server, can’t write HTML. An indic­ator that “things” need to get simpler.
• Vir­tual – the new real­ity for developers
• Secur­ity Focus on Applic­a­tion level Vir­tu­al­iz­a­tion