iCluster
The computer revolution again
Apple and the computer revolution
It is legendary how Apple revolutionized computing by making the computer to a home-device. There is no need to go into detail as how much the home computer has changed our lives in so many aspects, from gaming, to communication, to students, research, business and to empowering poor artists and composers to be able to do things previously only dreamed of, etc.etc. For myself, this has meant being able to write large-scale symphonic music and electro-acoustic music in ways I could not do without my home computer, bringing a full orchestra, recording – and mastering studio into my home, on my desk. My computer holds everything I need. Also I have been able to make fully photo-realistic abstract art in Photoshop, impossible to create with conventional methods, using only the paint-tools and a section of filters. Or making wonderful landscape animations in Bryce, or editing multi-camera concert movie in FCP, making professional DVD master in DVD Studio Pro, etc etc.
But not only in our homes… the desk-computer has also permeated business and school and research. Basically every business has desk-computers to do work on and what the desk-computer means for science and research is unfathomable.
But this is the past and to-date. Naturally the stand-alone desk-computers are always getting better and more versatile, OSX becomes better and the GUI both more beautiful and more task-efficient. But it this the end? Is the stand-alone desk-computer the end-all of affordable computer power? Does it fill all computing needs? Is it the only computing area needing of a revolution?
What’s next to revolutionize?
Supercomputers are still out there and they are still far away from the reach of most computer users. They are still needed and they are still the goal for many many users (more than one may think). But supercomputers are extremely costly to build, harbor and maintain. More have needs for various degrees of supercomputing than can afford it. SETI found a way around it by their famous “SETI at home” distributed computing solution. This way even more “obscure” research could be done, which otherwise never could afford the power of a supercomputer, neither to build or buy or even rent time on. But SETI’s model does not fit everyone. Some times one needs actual real-time computing power. So other solutions are needed.
Big Mac
So, what can you do if you can’t afford a “real” supercomputer, but need the power? Virginia Tech made the headlines – not only in the Mac community – with its “Big Mac”, a cluster of 1100 dual G5 processor Macs (i.e. 2200 processors). One of the up-sides of this Mac cluster supercomputer is – apart from the price – the time it takes to assemble. A dedicated supercomputer can take years to build. Ordering 1100 Macs and hooking them up takes much less time. Another up-side is the scalable property of a cluster of off-the-shelf-Macs. If you need more power, add more Macs. If one node drops, just unplug it and put in a new Mac. Very nice. The Big Mac was the third-fastest supercomputer and cost a fraction of a normal supercomputer.
Big Mac down side and cost efficiency
So, it cost a fraction of a “real” supercomputer. But there is also a down-side, which made it more expensive than it needed to be: what they paid for versus what they used. Each G5 had its large body housing components they didn’t use, taking lots of space that costs money. Each had a video board and 2 monitor ports (not used), sound board with analog I/O and optical I/O (not used), superdrive (not used), USB and Firewire ports (not used), extra hard-drive bays (I guess not used) and even a modem-port (not used) and PCI slots (not used)… So, here they where, paying for 1100 super-drives, video-boards, sound-boards and thousands of USB, Firewire, optical audio I/O, monitor ports, etc, that they did not use. Sure, the Big Mac cost a fraction of other supercomputers, but “a fraction” still meant 5 million USD! (They bought the G5s at 3000 USD each = 3.3 million. The total cost of the Big Mac was 5 million). 5 000 000 USD. Let that sink in… Five million dollars… most of it paying for components they don’t need and space to harbor the components they don’t need.
So, what did they use? The motherboard with RAM, the network board and Ethernet port. That’s it. Why pay for all these redundant components and redundant space? The space alone brings up the price of the cluster sky high. You need to have a dedicated very large room (in essence a hangar) to house it. And a pretty hefty cooler system that costs a bit. Soon after they had it up and running, Virginia Tech changed all the Macs to Mac servers for that reason… They “lost” a lot of money in the switch of selling off the 1100 Macs and instead replacing them with new G5 servers, but the cost in space motivated that swap. But still paying both for components they don’t need and space to harbor it.
Even many small companies (small research companies, construction engineers, animation artists, etc, etc) can have use of more computer power, but don’t have the money for loads of Mac Pros or Mac servers nor the space for them. Why pay for components you don’t use and thus taking more space than needed? What does the small 1-5 man animation studio do? What does the singular researcher do? I thought about clustering a pile of Mac Minis, but even there the price far outweighs the computing power as a node for me and it holds lots of components I don’t need but which greatly adds to the cost of the unit: you pay for a lot of components you don’t need at the expense of processor power that you do need. How about Mac Server? Well, for me, it is far outside the range of my pocket. And again, why would I need a video board or extra hard drive bays? Sure if you are a large university with eyes on building a new Big Mac or such, but me? A small 1 man business who needs to sop up Compressor? A small university department who needs some extra power on a strict budget? So what now? What can Apple do?
iCluster
Imagine a unit much the shape of a Mac Mini, but even smaller… No optical drive, no video- or soundboard, not video or audio or USB or Firewire ports, no modem… Instead it only holds a motherboard with a good processor, RAM memory, a small flash-memory or small hard drive for the system (or why not boot over network from the main controller computer?) and a network board with an ethernet-port: the iCluster! The cost for loosing the optical drive, modem, video, USB and firewire etc, can be put on a better processor instead. It does not cost much to produce and will totally revolutionize the idea of clusters and supercomputers. It is fully scalable. It takes a fraction of the space of even a Mac server, not to mention a Mac Pro. Do you need 2 to help Logic when using Sculpture and other processor-intensive plugs? Or 5 to greatly speed up Final Cut Studio Pro, Compressor or Vue de Esprit? Or need 3000 to do serious meteorologic or climate calculations? Need 50 for a small company render-farm? Or do complex chemical analysis if you a small research company? The scalability of the iCluster is beyond anything we can imagine!
To make it even better yet, sell the power unit (transformer) separate and with the ability to hook 5 or 10 iClusters to the same single transformer. This will also save space that one can have several iCluster nodes on one single power unit. Less heat, less space, less hassle with electric extensions.
This is the next step of the computer revolution. This is worthy of the statement “the first 30 years where just the beginning”.
Scalable power for all needs
So, an iCluster dedicated cluster-node out-of-the-box could be the next computer revolution. At a price of 500 – 1000 USD for a dedicated cluster node anyone – universities, single institutions, lonely professors, small scale animation companies and even private people like me – can buy and assemble their own cluster to budget and need. For the price of the 1100 G5s in the Big Mac Virginia Tech could have a cluster of well over 3000 iCluster nodes and still have money over. The cost of the server-room and cooling could be cut to a fraction. So counting in the lower price of server space and buying more iClusters for the money saved in space they could get the worlds by far fastest supercomputer. Think of the size… a Mac Pro or even a Mac Server vs a slimmer Mac Mini. The iCluster could be even smaller than the Mac Mini! The space of one Mac Pro (one single node) you can have – what – some 40 – 50 iClusters! Why build a large server room that costs loads to build and maintain, when you can the same node amount in a closet? Beginning to get the picture? The space alone will revolutionize super computing! You need cooling? One single home-sized air-conditioner could cool a small room full of several thousands iCluster nodes. Or have your render-farm of 200 iClusters in a cheep refrigerator! Get the picture? A small architect firm can have their own small render-farm to make large scale detailed animated renditions of projects, fitted on a single book-shelf! A small institution can have their own super computer to do bio-calculations or whatever, fitted in a spare toilet room. A small engineering contractor can do complex stress-calculations for building-projects using an iCluster stack in the desk drawer. A surgical unit can do detailed simulations in prep for an operation. A demolitions contractor can calculate or simulate explosions impacts for tearing down a building more safely. Things that normal-income students and free-lance whatevers can do is beyond my knowledge or even fantasy… how about your own home human-genome project? Amateur astronomers can do complex calculations of comets and other celestial bodies. What type of science projects can an average collage do with their own mini-supercomputer?
Now you may say that 10 or even 100 iClusters can’t compete with a supercomputer. You’re right. But everyone who needs more computing power doesn’t need a full supercomputer, just more computing power than they can afford or house today. Even a modest cluster can do wonders for many. A single institution can afford 10 or 40 or 100 iClusters but could not afford 1 hour on a super computer. Also they will have to wait for their que-time to come on a supercomputer they rent time on. Their own small iCluster cluster is not as fast, but can run around the clock for them so their task could be finished by the time their cue-time would arrive on a super-computer they wanted to hire time on… If several institutions at one university get a pile of 5 or 10 or 50 or 100 iClusters each, the whole university would have a nice number of nodes that can be used as a whole as well. It is totally scalable… each department have their nodes that they use, and when they are not using it, it can be added online as nodes for a larger cluster…
And what happens when the institution moves to new digs? How easy is it do dismantle, move and remantle a cluster of hundreds or a thousand Mac Pros? You could load the whole iCluster cluster into one large car or a van. Are you on an research expedition to do climate research on Antarctica? Need a supercomputer to crunch all the data? Bring 5 suitcases stuffed with iCluster nodes and you have a cluster that would be the envy of many climate researchers. Or if you just need to sop up Compressor or Logic or Bryce, you can buy 1 or 2 or 5 iClusters to make your work flow more smoothly. Or want the ultimate super computer? Cluster 3000 of these babies!
Super home computer
Now why would I need a cluster at home? Are you the proud owner of Poser and Bryce and wish to make your own 30 minute animation movie? But you realize that you need to lock up your one and only Mac Pro for a year, around-the-clock, just to render it to broadcast quality? As for me, I am a computer artist and composer. I want to make more complex and larger animations in Bryce or Vue than I can do now. As a composer I work in Logic. My synth of choice is Sculpture. It is a real processor-eater. I love to do complex automations of pans, volumes, filter-settings and various of Sculptures parameters for electro-acoustic music, which eats lots of processor power. Also my EWQL Symphony Orchestra prefers loads of power (the makers recommend 8 computers for it to run smoothly!). Logic and Vue have built-in clustering ability, but I can’t afford 7+ new Macs to cluster, and one new Mac Pro just doesn’t make enough difference to motivate the cost, nor do I have the space in my small apartment, nor do I need yet another full stand-alone computer with extra hard-drive bays, superdrive, etc. But I could use and afford and have space for a small stack of iClusers and it would make a huge difference in what I can produce both in Vue and Logic. Daytime I would use it for Logic and night time I would have it as a render-farm for Vue or Bryce.
“The first 30 years where just the beginning”
This would be a worthy way to celebrate Apples 30th year of revolutionizing the computing world. In one stroke, they can change the life of both universities, singular scientists and students, many small companies and many many many people like me: artists with dreams and visions and programs that vastly outrun the computer I can afford… We have the programs we need to do what we dream of, but not the computing power to actually make it happen all the way. Apps like Vue and Logic have outgrown the power of the singular desk-computer and large render-farms or clusters built of stand-alone desk-top computers are far out of reach for most of us. This is why they have clusterability. Simply moving from the G5 I had to an intel Mac made a great difference, but still is not by far large enough step for me in processing power that I need to finally make that large orchestral piece for Sculpture or the animation-film I want… Long before my singular Mac can render even 10 seconds of my animated film, I need the computer for other tasks… Same with rendering a single print size Bryce picture with volumetric atmosphere… it just takes too long time… and I cant let my Mac work around the clock for renders, because I need it for my active daily use. And in Logic, sometimes freezing tracks in Logic isn’t the solution, because the things I do in Sculpture are so complex that while I am composing the part in real-time, my computer freezes with “core audio not fast enough” even when that track is solo, so I can’t even finish composing that part, so there is nothing to freeze… I simply need more power… MUCH more power than simply upgrading to a new Mac Pro can do…
Wont iCluster compete with other Macs?
I understand that Apple does not want to compete with itself and may fear that an iCluster may steal buyers from the Mac Pro line. But that can’t happen. Keep in mind that the iCluster can never be used as a stand-alone computer: it is NOT a buffed up Mac Mini, it is NOT possible to hook a keyboard and monitor to it and run it as a computer. It can’t replace the Mac Server as a server either. It can _only_ be used to add processor power to your Mac, not replace it. You still need your iMac or Mac Pro as the master controlling computer for the nodes and to work on. So, there will not be any competition between the iCluster and the Mac. It will be a totally new product in its own right. And if I am not totally off, it will bring much much much more revenue to Apple by finding new customers. Think of it… first off, I can see all my friends who are composers and artists buying a few iClusters to buff up their studio, who would never afford to buy one more Mac Pro just as a node to buff up their computing power. Second, I can see a global “Big Mac” revolution, making iCluster a household name in larger scale clusters and super computing. How long time does it take to build a super computer? Well, with this sollusion it takes the time it takes to order the ammount of nodes you need and then just hook them up to your network… How long time does it take to move a super computer to new digs? Well, with the conventional build, it basically cant be done, but with this sollusion you can dissemble, move and reassemble the whole cluster in a matter of hours!
iCluster and iCluster Pro
And there is another thing Apple could think about… iCluster and iCluster Pro. Think iCluster as a simple low-lost 2 x Core2Duo (4 cores) for people like me who need a small render-farm and/or Logic power, et sim, costing some 400-500 USD. Then we have the iCluster Pro with e.g. 4 Quad-Core processors (16 cores) that will cost much more, but still be cheep for the serious supercomputer clusterers like universities or as render farms for larger animation companies, et sim. Let an iCluster Pro cost some 2000 USD. The university will still save loads of money on the fact that they can put some 40 iCluster Pro’s on the same space as one single Mac Pro. So, make a simple iCluster for home / small business use (small render farm, Logic nodes etc) and an iCluster Pro for the big-shot super computing guys.
Too bad I don’t own any Apple shares… I wish Apple will kindly give me a very modest royalty if they use this idea and develop the iCluster. But wont insist: the main thing is that it is made. I will pre-order a stack.
Conclusion
In checking for info about the Big Mac for writing this, I stumbled upon the following I snip from Wired:
“If Apple were to offer a scalable, high-density hardware solution, I would push hard for a platform switch,” said Patrick Gavin of the University of California at Santa Cruz Center for Biomolecular Science and Engineering. [...] “You don’t want your cluster taking up an entire room,” Gavin said. “You also don’t want it to consume bucket loads of power.”
Gavin jokingly said the lab had considered buying a bunch of G4 Cubes and stacking them sideways like bricks to make a computing “wall,” but the computer must be seated upright to work properly.”
-Quoted from http://www.wired.com/medtech/health/news/2002/02/50454
So, I guess I’m not the first to have this idea of a compact node from Apple. What I find strange and sad and frustrating is that seemingly no-one at Apple has had this idea. That is why I finally resort to writing this, in hope that it will reach Apple. If you who is reading this has some way into Apple, please forward this idea to them.
