Continuous Integration Matters
Recently I’ve been spending quite a bit of time trying to work out the best way to enable my team at work to be able to rapidly meet several impending deadlines for new projects. In this I have once again started looking at some agile practices that have been largely tossed to the sidelines. Some of the more important practices that I am currently trying to preach again are the old of code standards and unit testing. While there are many people who see these as barriers to progress, viewing them as yet more things they need to do, my hope is to perhaps give them the tools to be able to see the truth; that these practices exist to empower developers.
One of the core issues that I see day in a day out is that testing almost always takes 2-3 times longer than originally expected. This is by no means the tester’s fault though. It has been shown over and over again that following a waterfall style approach to development is in fact the most inefficient way of producing software. The reason for this is because when you put testing at the end of the process, you force everything to grind to a halt as you backtrack to fix bugs.
In fact, unit-tests are designed to help prevent this. Now while I won’t go into the specifics on what unit-tests are or why they are good, as that has been covered by many wiser men than myself, I will say that holding developers accountable for writing them is a key component to making this ecosystem work.
Enter Continuous Integration
This begs the question of, what exactly is continuous integration. To quote the venerable Martin Fowler:
Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
More specifically it means automating an incremental build system, that encourages developers to find and fix bugs as they create them. This is after all the moment that they are in the zone, working with that code and most likely to be able to react quickly, already emerged in the problem. Including some standards support to warn the developer when they are about to commit code the is needlessly complex, can provide warnings that they may want to re-factor a small bit before continuing.
The important part of this whole process is that the developer be made aware of the issue as close to immediately as possible. If you’re in a month long project and major testing doesn’t happen until the fourth week, then the likelihood bugs uncovered in that fourth week pushing out the deadline is increased. This is simply because it is harder to go back to code that you previously wrote, than it is to fix it immediately after writing it.
Not to say that you will find every problem up front, but I can say from experience that when ever I spend the time to write unit tests and verify those tests before I push code, testing time is almost always miniscule in comparison to when I do not.
A recent example comes to mind. I was writing a small rules engine that would based on some simple conditions allow a CMS to control, based on dynamic input, whether-or-not a small bit of code would show up in the page. During this process, I did not do my due diligence and write my unit tests as needed. Because of this, some simple rules went back and forth with the testers for 3 days.
Finally, realizing my error, I stopped, wrote out a set of 20 test cases for how I expected the rules to behave, ran them, found five failures. Spent 20-minutes working them out and finally got my code to stable. At this point, not only was I confident that the code would work, but it went through the QA process with flying colors. Moral of the story, had I done the right thing up front and spent that 20-minutes working out the test cases, I would have saved 3-days of time while bugs bounced back and forth from me to the testers and back.
Continuous Integration is about keeping developers accountable for doing these things and verifying that the code they write doesn’t break someone else’s code before it makes it to the testing phase.
When does it take to start?
In generally you need three things to start using continuous integration in your development cycle.
The first is the simplest, version control. This can be anything from GIT, to Subversion, all the way to CVS. The latter having fallen out of fashion for various reasons, though still in use. No matter what, every project and team should be using some sort of version control. It is your contingency plan if errors occur. It is how you find out what changed and who changed it.
Second we have the CI server itself. This can be either easy or difficult, depending on your platform. .Net developers have Team Foundation Server available to them, which can be configured to provide this type of service automatically. The specific server is called Team Build, which will keep track of build successes and failures and help keep the developer informed of problems.
A more general solution for the rest of us developers is Cruise Control. This server has various plugins that allow it to work for Java, Ruby, Python, PHP and just about any other language out there. It contains reporting tools and very tight integration with Subversion. If you are in need of a solid, quick start package, this is where I would start looking.
Lastly, you need someone to keep the developers motivated and encouraged to keep following these practices. It is very easy to “forget” to create a build process for the new application that you’re developing. It can definitely seem like more of an obstacle than an option if you are only looking at the immediate gains (though I argue confidence is a huge immediate gain). So, having someone (or a group of someones) to keep people excited and motivated to use these tools.
Actively show the successes that they have provided. Gather some metrics that show how many bugs were avoided in the testing phase, because of the incremental builds. How testing time has decrease. How production bugs have gone down. All of these things are the benefits of failing early.
When should I start?
Right now!
Whether you are starting a new project, or deep in the trenches of an old one, if nothing else, then start writing unit tests. However, taking a few hours to set up Cruise Control or some other build server will be paid back in full very quickly. If you have the server set up, then make sure when you create every new project that along with setting up your repository, that creating an automated build is on your list of first things done.
If you can have this set up and ready to go before you write your first line of code, then you are in a rock-solid position. Just remember, it is an active process. Continuous integration is a way to reinforce the good habits, it does not replace them. It is up to every developer to do his part.
Zend Server CE and Framework, VHost OS X Dev
So lately I’ve thought about switching my PHP development over to Zend Framework. The main reason for this is that I am just tired of rewriting the wheel. Most of the common problems with developing a PHP application have been solved and for once it would be nice to just build an application, instead of building a framework.
To get started in my efforts, I needed to set myself up with a local development environment. While I do have a production web server hosted with Slice Host (which this website is served from) and a local server at home, I wanted something that was a little less configuration heavy. After looking into several Mac OS X solutions, like XAMPP and Entropy PHP, I ran into Zend Server Community Edition. The first two, while decent solutions, were no where near as complete as the Zend provided solution. It was a clear winner for me.
Fast forward a bit, after downloading, installing and running the Zend Server CE, I was up and running. Much more quickly than I had anticipated actually. Now, the server control panel not only has modules predefined for most of the most common php suspects, it also has several already available that are a little more exotic, like DBLib (for accessing MSSQL servers with PDO). I initially had some questions on how to ensure the MySQL database was working correctly. After consulting a great article on setting up Zend CE in Leopard, I was off and running.
I do suggest one change to his post. In order to have access to the zend command line tools for Zend Framework you path should be adjusted like this in your ~/.bash_profile file:
PATH="/usr/local/zend/bin:/usr/local/zend/mysql/bin:/usr/local/zend/share/ZendFramework/bin:$PATH"
After changing this, be sure to restart your terminal window. You should now have access to the zf.sh file from where ever you are.
Adding Development VHosts
Personally I prefer to use fake local domains for my development. I prefer to type something like http://mysite.local/ in my browser than http://localhost:8181/, which is just non-descriptive and difficult to remember. Especially being that I tend to juggle projects. So, I created some simple instructions for making the Zend Server aware of your local development environment.
My personal taste is to reuse the /Users/Drew/Sites/ directory, as this keeps the information that I want my Time Machine to back up in an easy to find place. Plus it just seem more proper to edit and work with files there, than in the /usr/local/zend/ directory. The first step is to create an apache config file to help find our vhost entries.
$ vim /usr/local/zend/apache2/conf.d/vhosts.conf
Now insert this into that file. Of course change the username to be your own, not mine (Drew).
NameVirtualHost *:80 Include /Users/Drew/Sites/vhosts/*.conf
This will tell Apache to look for any vhost files we create in our home directory to parse. So let’s create a site.
Creating A New Zend Framework Site
Now, assuming that you followed my recommendation for adding Zend Framework into the path, the following should run without any issues.
$ cd ~/Sites $ zf.sh create project mysite $ rm mysite/public/.htaccess
I added the last line in, because we do not need to standard htaccess, as we will be doing this ourselves. Now, let’s create our vhost, so the app can be found by Apache.
$ vim /Users/Drew/Sites/vhosts/mysite.conf
Now insert the following into the file and be sure to adjust the username again.
<VirtualHost *:80>
ServerName mysite.local
DocumentRoot "/Users/Drew/Sites/mysite/public"
SetEnv APPLICATION_ENV development
RewriteEngine off
<Directory /Users/Drew/Sites/mysite/public>
Order deny,allow
Allow from all
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} -s [OR]
RewriteCond %{REQUEST_FILENAME} -l [OR]
RewriteCond %{REQUEST_FILENAME} -d
RewriteRule ^.*$ - [NC,L]
RewriteRule ^.*$ /index.php [NC,L]
</Directory>
</VirtualHost>
Now, I’m assuming the url we’ll be using is http://mysite.local/. In reality this could be anything you want, as our next step is to actually define it. So feel free to change it. In this case I’ve decided to keep the Mac convention of defining local urls with the .local postfix. So, let’s edit the hosts file and add your new host in, so the browser will know how to find it.
$ vim /etc/hosts
Edit the local line and add your url to the end, like so:
127.0.0.1 localhost mysite.local
And finally, we finish it off by restarting Apache:
$ sudo zendctl.sh restart-apache /usr/local/zend/bin/apachectl stop [OK] /usr/local/zend/bin/apachectl start [OK]
And that’s it! You should now be able to go to http://mysite.local in your browser and see your new Zend Framework site ready to go. You can now just start these instructions from the site creation point next time, to set up a new site. Now, you could put all the vhosts in the same file, but I prefer having them in separate files for easier management. I’ll likely be posting some more about using Zend Framework in the near future as I start to learn it and hopefully come up with some tips and tricks. For now though, if you need a resource for getting started, try Zend’s Quickstart Guide.
BoxeeBox hardware announced and the Boxee beta!
Amidst all of the CES buzz, there was an announcement about my most recent obsession, Boxee and the Boxee Box. It seems that not only is the Boxee beta open to everyone now, but that the BoxeeBox itself is to have quite abit of power under the hood. Those who know me have had the unfortunate luck of having to hear me droll on and on about how much I love the up coming BoxeeBox.
Until today, all I had been waiting to hear was some news the hardware in the thing would be able to support 1080P HD video. In fact, now that the Tegra2 processor from Nvidia has been announced as the engine behind this little beast, I’m ready to put my name on the pre-order list for this little bad boy.
According to Engadget, the Tegra2 will be:
The new processor is a 40nm chip that’s supposedly much lower power than Atom and even Snapdragon, and can still do full HD streaming and HDMI out. Supported OS’s include Windows CE, Windows Mobile and Android, with Chrome OS on the drawing board — “hundreds” of test systems are out there, and devs can order their own test units this month.
And the press release from Nvidia on the same page:
Consumers have been waiting for a truly portable, high-resolution, no-compromise Internet experience. NVIDIA’s new Tegra processor delivers that by combining lightning-quick browsing, streaming 1080p video and Adobe® Flash® Player 10.1 acceleration with an immersive 3D user interface and days of battery life.
Overall, this gives me hope that this device will finally ween me away from using my PS3 and having to transcode my high-resolution video through PS3MediaServer. While this has been a process that has worked well (for the most part) there are some issues with the process. A few videos play perfectly on their own, with a compatible player, but when transcoded will crash after several minutes. While these are most likely due to errors in the video files themselves, it is one that is not as apparent when played directly. The other issue is that transcoded videos cannot be searched through as easily as flat files.
These are limitations that are for the most part annoyances. However when coupled with my need to run Windows just so I can have PlayOn available to stream Netflix and Hulu to my PS3, the BoxeeBox becomes an even sexier choice.
What about Boxee itself?
Boxee itself is a really compelling piece of software. Not only does it provide a platform for me to play my video library, but more importantly, it is a full portal to an entire world of streamed Internet content. These are done through Boxee’s robust and very open app platform. A quick look through the beta shows just under 2 dozen current apps, including:
- Netflix - Online movie rentals
- CNN - News and world events
- Crackle - Sony’s online video service
- MLB.TV - Major League Baseball
- Pandora - Internet radio
- Last.fm - Internet radio
- Revision3 - Internet Television
- Flickr - Photo services
- MTV - Do I really need to say it?
- And many more
These are just the current launch offerings. The best part about Boxee to me, compared to its competitors, is that it is built with an open app platform. It is designed to encourage content providers to utilize the device in the way they want to. Meaning, you could subscribe to a paid service (like you do with Netflix) or a provider could build their app so that it is ad supported. This to me encourages more content to be delivered through this platform in the future as it is proven to be just as valuable a business model as broadcast television itself.
One little hidden gem that I found is that there is built-in support for finding and watching semi-current television. This is done by aggregating several sources, such as ABC, NBC, CBS, Fox and even Hulu. This is all tucked away under the TV Show Library and has a very fluid interface for locating shows that are available and picking not only the episode to watch, but its source too.
In fact, I was quite surprised to see Hulu already supported under Boxee, after the huge trouble between them earlier last year. While the support itself is relatively limited, as you cannot search Hulu directly, the ability for them to create a full app later on that can be ad supported gives me hope that we will see full availability to this content. As soon as they can prove to their content providers that there is money to be made here that is.
Overall, I am just gushing about this announcement. Getting this device will finally finish my home theater media project, at least when it comes to hardware. According to D-Link, the BoxeeBox itself will be under $200, which is steal compared to the other HD quality streaming devices on the market. This is going to be a great year for digital media and internet enabled television. In fact, this will likely be the year that I drop cable completely.
As soon as I get my hands on the device (which may be several months away), I will be sure to given a full review of the setup and performance.
Merry Christmas to All
I know it is politically correct to say, “Happy Holidays,” however I stand by the fact that Christmas is a government holiday and is mostly a commercial holiday to me anyway. So with that, I hope everyone properly stimulated the economy today and got all sorts of fun toys.
I wish your holiday season to be a happy and safe one. Full of good food and fun times.
New Domain and Updated Theme
When I decided to try blogging again, it started as mostly an experiment in getting to know WordPress. To that end, I did not put alot of time or effort into thing like picking a domain name or styling the site. Mostly it just seemed a good use for the .name domain I had been holding onto for a couple years. Over the last few months, while I haven’t blogged a whole bunch, I do feel as if I would like to grow beyond a personal brand and perhaps try taking it a bit more seriously.
To that end I bring you, Abstracted Method.
It is not 100% complete, but already I have refined the template beyond its humble beginnings. I kept much of the layout of the previous design as I do feel it important to focus on the content more than anything else.
Anyway, my new goal is to start writing some real content at least once a week. At the same time I will be broadening my topics a little bit to include my movie / television geekdom. Mostly, this site will be an abstract of me. Of the things that catch my attention and drive my focus.
Creating A Dynamic DNS Script With Slicehost
Note: The script used here is written in Python. I did not write this script. However, in the time honored tradition of the Internet I am going to reiterate it here, as it was a little hard to find a solution via Google. The original script came from a Slicehost forum poster. I just want to make clear that I am not the author, just a fan of his work.
In all of my new server craze, I’ve come to have a need to be able to connect to it from where ever I am. Initially I thought that the best solution for this was to just use the DynDNS.com free service. It does exactly what I need with three exceptions.
- I have to use one of their canned domain names.
- After 1 month of inactivity, they will cancel your account.
- My current AirPort Extreme router does not support dyndns.com updating like my old LinkSys one did.
Now, most of these are either easily overcome or downright nit-picky. #2 and #3 could be easily overcome with a script that updates the IP address periodically. However if I’m going to need a script for this, I might as well go to the next step and use a solution that let’s me take care of #1 as well. Enter SliceHost.
I’ve been a very happy customer of SliceHost for a little over two years at this point. I started with a Gentoo slice and am currently rocking a Ubuntu install, simply so there is a little more fire and forget happening. I recently started using their wonderful DNS management system to add a post.drewbutler.name to point over at the Posterous servers. So I figured why not do the same thing but point it at my home server.
Setting Up A Type A Record On A Zone
In network terminology, your DNS is split into zones and records. The zone is the main domain name itself (drewbutler.name.) and the record is essentially the sub-domain (home). Now, I am going to say that this is an extreme simplification, however for our purposes, it is relatively accurate. For more in-depth information, please see the Wikipedia article related to DNS, zones and records.
I will assume for this exercise that you have your domain (zone) already set up and that we are simply adding a new sub-domain (record) to it. In the SliceManager, go to DNS -> Domains and click the Records link next to your domain. Once you’re in the records list for your zone, click on the New Record link. We’re going to set things similar to the image below, though substitute the Name field with your sub-domain and the data with your home IP address. You can use the dyndns checkip service to easily get your current ip address.
Hit update and you should be able to go to home.yourdomain.com and see your own server (assuming you have a web server set up). Otherwise, if you have a terminal available, run: ping -c 1 home.yourdomain.com and you should see your home ip address appear in parenthesis.
While we’re in the DNS records, you should see your new subdomain “home” listed. Hover over the edit link and copy the url there. It should be something like this:
https://manage.slicehost.com/zones/1234/records/123456/edit
The first number is your zone id and the second is the record id. Write down the record id (in this example, 123456) as we’ll need that in a little bit.
Using the SliceHost API to do some periodic updates for us.
First if you haven’t done so, let’s enable the API and/or get the API key. Go to the Account -> API Access in your SliceManager and click Enable API Access (if applicable). Now you should see you API password displayed. Copy and paste it into a document, write it down or whatever; we’ll need this in a bit as well.
For this next bit I am going to assume you are running some sort of *nix server and have 2 things installed, Python and Subversion. If you do not have them installed, please do so now. Both are very common packages and the installation should be very well documented for your distribution of Linux.
The script makes use of a great package called PyActiveResource. This is essentially a port of the ActiveResource module for Ruby, but in Python. At this moment, it seems the best way to get this package installed is via subversion, so let’s do that.
svn checkout http://pyactiveresource.googlecode.com/svn/trunk/ pyactiveresource-read-only
cd pyactiveresource-read-only
sudo python setup.py install
Assuming there were no errors, this should be installed as a python module now.
Next let’s setup a script to run. I’m gonna assume you are running Vim like I am, but please feel free to substitute that as appropriate.
sudo vim /home/drew/slicehost_dyndns.py
/home/drew/slicehost_dyndns.py
import urllib, re, sys, os
from pyactiveresource.activeresource import ActiveResource
api_key = 'xxxxxx'
api_url = 'https://%s@api.slicehost.com/' % api_key
record_id = '123456'
class Record(ActiveResource):
_site = api_url
results = Record.find(id=record_id)
if len(results) != 1:
print "Can't find Record %s via SliceHost API." % record_id
sys.exit(1)
salon = results[0]
found_ip = (re.findall('[0-9.]+', urllib.urlopen('http://checkip.dyndns.org/').read())[-1])
if salon.data != found_ip:
salon.data = found_ip
salon.save()
print "IP Updated: " + found_ip
else:
print "IP Unchanged: " + salon.data
Update the api_key and record_id with the information we wrote down before and save it. Be sure to maintain the spacing on lines as best as possible. Python is very strict about the use of white-space. The script from the forum post was actually not spaced properly and therefore didn’t run. That is really the only change I made. Otherwise it worked perfectly.
From the command prompt, you should now be able to run
sudo /home/drew/slicehost_dyndns.py
And will see either IP Updated or IP Unchanged. I suggest testing it at least once before moving forward.
crontab -e
Add this to have the script run every 10-minutes. As the script only does an actual update if the ip-address changes, it is fairly innocuous.
*/10 * * * * python /home/drew/slicehost_dyndns.py
You now have a working dynamic dns setup for your home server. I’ll be working on some updates to the script to do things like log when it actually changes the dns entry. I figure it’ll be helpful to monitor who often the script really needs to run and if there are any patterns in when the IP address expires. Might help in customizing the cron’s timing and thus decreasing load on the Slicehost servers. As it stands now though, this should be a very low impact solution, so I doubt they’ll notice.
An Easier Way To Keep WordPress Up To Date
Lately there has been quite a stir at my job regarding WordPress. In this we have played alot with deployment methods, including proxying it through another application to add an extra layer of features. Because of all of this I have had quite a few conversations with teammates about the easiest way to set up a new blog. As I feel I have a very quick and easy way to not only install a new WordPress installation, but also to keep it up to date, I figured I would share it here, in case that knowledge might be helpful to someone else.
A quick warning first, while nothing I am about to describe is necessarily difficult nor hazardous, all of it requires that you have some sort of direct access to your server. In the case of my explanations, I will be speaking of doing this through an SSH terminal connection. Some familiarity and access to Subversion is also needed. Mostly, this is a solution that should appeal to other developers.
Alright, on with it then.
When you first go to install WordPress, you are generally directed to download a nice zip archive of the software. This is generally accepted as an extremely easy way to install WordPress itself, however the trouble with this method comes in later on, when you need to update your installation to a newer version to combat the inevitable bugs that plague all software. While WordPress does a great job of automating most of the process, you are generally advised to back up all of your themes and plug-ins, which you will need to reapply afterward, as the update process will many times over-write some of these files.
The second issues come in from developers like myself who may or may not tweak the WordPress system itself, to perhaps change how tags are encoded. These changes generally will have to be completely redone when a new version comes out. They will be over-written by new versions of the files.
I’m a developer, so let’s start thinking like a developer.
Under the download section of the WordPress site, let’s go to the Subversion access section. Just as they assume you have Subversion installed, so will I. If not, checkout the Subversion website for information on installing it.
For the most part, we are going to follow all the instructions they have here, except for the part on which repository to use. They link to the trunk of their repository, which while generally stable, is still a development version and could have stability or performance issues. As of this writing, the current stable version of WordPress is 2.8.6, so that will be what we use in these examples. However, you can just replace the version number in the examples with whatever is current to stay up to date and avoid installing older versions.
Let’s begin.
From the command-line of the server:
# Go to the directory that your WordPress site will live.
$ cd /var/www/mysite
# Check out WordPress into the current directory
$ svn co http://core.svn.wordpress.org/tags/2.8.6/ .
That’s it, as I said, nothing difficult. The rest of the install process is just like in the Famous 5-Minute Install, only start with #2.
Now, what we have done here is basically checkout the repository as if we were going to be programming in it. Normally a developer would make changes and commit them. We definitely won’t be following that part of the life-cycle. However, we now gain all the benefits of allowing Subversion to manage when changes occur, so when we need to update, it can tell us when conflicts occur and as developers, we can manually merge them.
I still strongly suggest you keep backups of your site in case you break something really bad, however these will be for worst case scenarios, hard disk crashed or I rm -Rf /. my whole computer kind of issues.
Keeping up to date
In order to update to a new version of WordPress, now all we need to do is use the switch operation in subversion to switch the tag you have checked out.
# Go to the directory that your WordPress site will live.
$ cd /var/www/mysite
# Check out WordPress into the current directory
$ svn sw http://core.svn.wordpress.org/tags/2.8.7/ .
While as of this writing that is not a valid version of WordPress, one day when it is, that will be the line I use to upgrade to it. Of course just change the version number to match whatever is current. An update from 2.8.5 to 2.8.6 yielded the following output
$ svn sw http://core.svn.wordpress.org/tags/2.8.6/ .
U wp-includes/version.php
UU wp-includes/js/swfupload/plugins/swfupload.speed.js
U wp-includes/functions.php
U wp-includes/formatting.php
U readme.html
U wp-admin/press-this.php
Updated to revision 12288.
Were there to have been a conflict, due to changes would have otherwise been overridden by this update, there would be a ‘C’ next to the file name, instead of a ‘U’ which just means ‘update.’ In that case you would look inside that file and you’ll see both versions of the code inside there and you can then decide how to repair it. In most cases, unless you make changes to the WordPress core though, you will never see this.
This is what I consider the worlds most painless WordPress update, as when a new version comes out, I simply switch to it and the log into the /wp-admin. It will have me update the database, as it does and I am done. Takes less than a minute to update in most cases.
Anyway, hope that is helpful. While I’m sure this is a bit much for many people, at least for developers, it is fairly fluid with our normal process and is by far the ideal way to stay up to date.
Google Chrome OS: Is Not Enough, Enough?
No one can deny the effect that Google has had over how we use computers in the last decade. In the beginning it started with a simpler home page, one less burdened by the clutter that other search engine’s provided. From search it moved to dominate the web advertising with it’s Adsense program that provides relevant marketing to web pages and the ever-changing content of the web. They sought to change the face of e-mail first by giving you a simpler way to organize and manage your in-box. They also looked to offer free online tools to write and manage documents that were more traditionally handled by expensive word processors. More recently they have sought to change the face of e-mail and general communication again, with their newest beta product Wave.
These are only a small number of the growing projects that Google has in its labs. Lesser known to some has been Google’s dabbling in building its own operating systems and browser. It started with mobile phone platform known as Android that has taken the developer and general geek communities by storm, promising a truly open platform in the ever-growing mobile universe. This was a complete affront to the blossoming Apple iPhone movement, which while exploding, is somewhat plagued by sense that big brother is always watching and stifling innovation. The promise from Google was that it wanted to facilitate innovation by providing a truly open alternative for the traditionally very closed mobile market.
Soon after they released their own browser, Chrome. By their own admission it was intended as a proof of concept for how to build a better browser, one they planned to keep open-source (like many of its competitors) and that would show Google’s intent to smooth out some of the ripples in the normally choppy waters of the web. It is in their best interest after all that every one be using a browser that has good support for current standards and that is fast and secure enough for people to feel safe in moving their lives online; into Google’s world. That is their business model after all. They are cloud computing for the masses.
Recently Google released a second operating system, one that has had many in a buzz, wondering what they were up to. Until very recently, all we really knew was that they were working on a new OS called Chrome and that it would be what could be considered a web-centric platform. What this meant to most of us was that Google was once again throwing away everything and starting from scratch, claiming they knew better. I’ll admit, I’ve been very wary about this new entry, looking at it from my own personal computing perspective and asking why?
So, what is Chrome then?
Let us start by answer the inverse and stating what it is not. Chromium is not a traditional desktop operating system. It is not meant as a platform for which you are to install software like games and productivity suites. It does not have a start menu or control panel. Its purpose is quite simply to put the things you traditionally do on your home computer out into the web. In short, Chromium will not replace your desktop anytime soon. That isn’t what it’s for.
In the last week Google released the beta for Chrome and while I have downloaded it and loaded it up into a virtual computer, I must say, I was underwhelmed. Chrome is quite simply a fancier version of its Chrome browser just replacing your whole desktop experience. You run programs from the browser, instead of a start menu, you have a glorified bookmark bar of your most often used sites. Looking at it and then looking at my own triple screen computer setup, makes me cringe. How am I supposed to do everything I do in just a browser window?
Simply put, I’m not.
Chrome is not meant for my desktop set-up. It isn’t meant as a development platform or for gaming computers or for people needing the full office productivity suite. What it is meant for is a fast and secure solution for a very new, but emergent market: the netbook. Netbooks and other small, low power devices are taking over for every one from the traditional alpha-geeks to home users. Why? Because they are small focused devices that allow us to quickly get on the Internet. In fact, their biggest hindrance thus far has been that device manufacturers have tried to shoe-horn traditional desktop operating system on these devices. While they work, they are far from ideal solutions.
This is where Google is attacking and I think they will win a spot in this space very quickly. Because when you use one of these devices, you do so to surf the web, check your e-mail, look up contacts, read some news or any other number of online activities. If you need to work on a paper for school and you have one of these, well that’s what Google Docs is for. On the little atom processors these devices run on, I can promise that it will run faster than any of the desktop solutions. With their 9-10” screens, they are far from ideal multi-tasking computers anyway, so the simplicity can be a plus. As these systems are not meant to be your primary computers, the idea of local storage becomes unappealing, because then you have to worry about syncing devices later on.
If it is all out in the cloud, then your data follows you. In fact, if you drop your netbook or it gets stolen, because there is minimal data on it (as long as you didn’t write your username and password on the thing), it is still safe. You can buy a replacement, log in and voilà, you have your data back. This is the future of computing that Google is trying to help enable.
Is it secret is it safe?
According to Google, Chrome has been built from the ground up with security in mind. When it comes to malware like viruses, keyloggers and spyware, Chrome should be relatively safe based on the fact that you are not meant to install software on it. Traditionally the number one way to get a virus is for a user to install it themselves. Generally unknowingly of course. However with a system that is kept up to date and by using a secure browser, the likelihood of getting malware installed on your computer accidentally is infinitesimal.
Not to say that holes will not be found or that it will be 100% unhackable. Those are both impossibilities. However, by removing a lot of the moving parts that allow for infestation and keeping it simple, Google has much less to worry about security-wise. Really, the biggest threat for users will likely be phishing scams which are more about social engineering than infecting computers.
While your data not being on your computer also means that it remains more secure against an exploit that is found, there are plenty of people who look at keeping your data online as having other security implications. Such as your data is one password away from being stolen. That is a completely separate discussion though.
What’s left for the desktop?
For now I don’t think Microsoft has much to worry about. Windows is not Chrome’s target as a replacement. At least not in the desktop space where Microsoft makes their money. As I am excited to see where Android will take the mobile market, I am equally excited for Chrome. It promises to help focus how we interact with the web and to make these smaller computers just a bit simpler. In fact, just like in the beginning, when all the “experts” derided then for their barren homepage, with just a search box, Google is once again going back to basics and asking: what is it exactly that you are doing with your computer that needs it to be so complicated?
Chrome is not meant to be a 100% solution. It is meant to take what people spend 80% of their time doing on a computer and do it better. I think in that it very well may end up being a success. Unlike its competitors though, Google doesn’t win by getting you to use Chrome (it is free as in beer after all). They win by switching the paradigm away from desktop-centric computing to cloud-based computing, something any true competitor to Chrome will do for them as well.
New Gentoo Media Server Nears Completion
A little while back I discussed my (at the time) recent purchase of an Acer H340 Windows Home Server. At that time I was just getting into ripping my DVDs into x264 / DIVX formatted video files. At about 1.5-gigabyte per movie, it seemed like it would take me forever to fill up the 4-terabytes of space the server afforded me. I figured by the time that happened storage would be more dense and a new, similar solution would be available to replace my “aging” H340 server. Of course, as usual, I underestimated my enthusiasm in filling the device. See, shortly after I began this project I started modifying what I was filling my library with.
With my discovery of the PS3MediaServer package, I found serenity in being able to transcode higher resolution rips. Now all the sudden a whole new world of ripped 720p and 1080p BluRay media opened to me. This of course went great for a while. I was able to watch all sorts of great movies on my nice LCD TV in uber high-resolution and my little media server was chugging away, doing its best to keep up with all the transcoding I threw its way. Soon, the 2-terabytes that drive duplication afforded me filled up, so I deduplicated my TV show archive and made my way to 3 -terabytes. Which I of course filled equally fast.
After removing almost all duplication, I started to notice other issues. Every once in a while the server would just hang while trying to transfer data to it. At first it was very occasionally; about once every 2-3 days. Unfortunately, soon this mutated into every time I would do so, within 5-minutes of starting a copy job.
Around the same time, I started noticing that the server would hang anytime I would remote desktop into it. It wouldn’t happen as soon as I entered it, however usually as soon as I tried to open explorer, the whole UI would just hang. Needless to say, after a few days of this, I just shut it off. It was clear I had exceeded the capabilities of the machine. I think the extreme amount of data and media that I was asking it to index was just simply causing the little Atom processor too much trouble. Causing kernel panics and the like.
I still stand by that it is a great little server, however I think I tried to do too much with it. It intended as a backup server that also could share some files. However as a full media server, it just did not have the processing power to keep up.
A New Solution
This led me with a huge gap to fill. So after a couple weeks of looking around and trying to convince myself that I did not need to spend the money to buy a full-out server, I did anyway. I did not go with 2-terabyte drives, as I am always wary with the largest hard disk size, however I did decide to go with 1.5-terabyte drives. Seven of them to be exact. I currently have 10.5 terabytes in a RAID 5 array, giving me approximately 9-terabytes of space (minus the file-system overhead of course). I tried not to go too overboard. I bought a modest dual-core processor with 4-gigs of RAM. I really only splurged on the hard disks and the case itself. As I would be running this constantly, I needed to be sure the hard disks remained cool, so fans and space were a real consideration.
I decided on the Supermicro CSE-743T-645B Pedestal Chassis, with eight hot-swappable hard disk bays. The goal being that if a drive dies, I want to be able to replace it easily and start the recovery process right away. The case is a little obtuse to work with and had I researched a little better I would have probably gone with a Supermicro server motherboard. Being that all the connectors (fans, front control panel) are very specific to their hardware, one of their boards would have been a more direct fit. However with a bit of research and common sense (and the inclusion of a fan controller) I was able to get it all up and running with a cheaper Asus motherboard.
While I did jump back on forth on Linux distros: from Debian Lenny, to Ubuntu, finally settling into my old favorite Gentoo. It required a bit more work to get it set up, however in the end I truly believe you are more familiar with you system and what is installed and how to make changes because of the extra set up time. Currently I have not done much with it aside from setting up my hard disk array.
Next Steps
After a little research, it seems that I will be switching from PS3MediaServer to the MediaTomb. As MediaTomb runs as a service, it will integrate better into a Linux server, because it will be available as soon as I turn the machine on. The lower footprint of the software will also be advantageous, leaving the server to do other fun things, like acting as a torrent box.
Once I get a little further I will share the setup of MediaTomb here, as from what I can tell, it is a little tricky under Gentoo. Until then though, I will be copying data from the old system onto the new server. I’m doing it through a CIFS share, from the server, so hopefully it will be a little lighter-weight on the poor Atom processor.
I still highly recommend the H340 as a Windows backup solution. It was very easy to set up. However as a large media server, it fails miserably. Had I known when I bought it that I’d be dealing with so many more movies, I would have probably skipped that step. I am hopeful though that I will be able to sell the little server to someone who can give it a more proper home.
With the H340, I feel like I got a puppy because it was cute and then made it plow the fields out back. It didn’t survive very long out there.
Bring in the spam!
Over the last several months, since I started writing this blog I’ve noticed a tiny trickle of visitors. Most seem interested in the Acer H340 that a blogged about in my first post. I felt bad for many of them, seeing their search keywords, knowing my little bit of writing wouldn’t help their plights. The only consolation to this has been that I knew adding the link to whsplus.com at the bottom of the article had been a smart move, as it seemed to help those poor folks out.
However, for the longest time I received almost no comments on any of my articles. While I would love nothing more than to see readers talking and asking questions in my comments area, I knew that not only have I failed to advertise this site, but that I really do not have any real compelling content as of yet. After all, this is nothing more than a personal side project.
Beginning when I changed the site’s name a month back I started to notice another change. I started to see some comments come through. The first was a classic bit of spam, all filled with links to various sites. Akismet caught it though, so no worries there. Then I noticed several others that, while short, seemed to be written by people. Finally living breathing people reading my site!! Or perhaps not.
It seems that comment spam has entered a new realm. Now spammers are smart enough to know that they can’t just add in a bunch of links and hope the comment makes it through. They need to seem like they actually are responding to your post.
In these cases I noticed a few things that gave it away.
Anyway, so I am officially locking down comments and will be looking for real visitors along with them. This is mostly because I do not want to help people scam others out of money by increasing a blogs influence rate in search rankings.
Anyone have any tips for weeding out blog spam they’d like to share?