Blog

Wanna be Impressed? Check out Google Analytics!

AustinMash! has been live for about 4 months now. From the beginning I included code in each page of the site that recorded every time the page was loaded into a browser, along with the user’s IP address and the current date. This was primarily set up to support the live site stats on the PixelList page, which shows in real time how many unique visitors visited the site today, this week, this month, and the total since launch (over 2400 people now – whoop!). It also shows the same stats for ad clicks. I also created a small desktop utility so I can watch the numbers climb (way fun!) and I have a private report page set up that shows the raw data in an easy-to-scan format.

Although this setup worked great for its intended purpose, it left a lot of unanswered questions related to what users were doing on the site. For example, I started to see, in the report page, a large number of page views from the same person. I mean, page views in the 60-90s in the same day, and again the next day, and the next. Since there are only about 40 pages in the whole site, this was pretty weird. I can appreciate someone being so mesmerized by my site they keep re-reading everything, but come-on, over 200 page views in 3 days? I had to go dig into the server log files to see that this user was not going through all the content at all, all requests were the same, for the blog home page! The logs also showed all the requests were being referred to from BlogSlides . Aha! I had recently added my blog to their index of blogs, and their unique feature is that they automatically cycle through a series of blogs in the same browser window, hence the name BlogSlides. Sure enough, the logs showed this visitor loaded the main blog page at regular intervals all day long. Ok, so this minor mystery turned out to be someone who left his BlogSlides window running, and by doing so created a bunch of bogus requests for all the blogs in his slide show, not just mine, providing another example of the unintended consequences of someone trying something new. Anyway, I found my answer through tedious scanning of my log files, not through my automated visitor counters. This would have been a lot easier with dedicated Web Site Analysis software.

So, I knew my hit and click counters were not good at providing the kind of deep analysis you can get from web analytics software, they were never intended to be, and creating such a system was never a reason for AustinMash!’s existence. All along, I’ve thought that if detailed visitor stats ever became important to me, I would have to shell out several hundred dollars for a decent and robust solution, something like WebTrends .

Then, a little while ago, an interesting e-mail appeared in my inbox – it was an invitation from Google to try out their new Analytics solution, which was, of course, free! Apparently I had asked for an invitation some time ago, probably when I set up my AdSense advertising account, and then forgot all about it. So, here was Google, one of the most impressive software developers of today (Google Maps, Google Finance, Google Spreadsheet, need I go on?) inviting me to try their latest creation. Guess how long it took me to add their tracking code to my pages? Not very, I’ll tell you that right now.

The hardest part in adding their code (about 4 lines of JavaScript) was in deciding where it should go. In my case, since I have made the commitment to have the Skyline appear on all publicly accessible pages, it made sense to put the Google code next to the code that generates the Skyline Ads area map. Although Google says it may take a day, within a few hours of uploading the new code to the site my personal Analytics page was showing visitor data, and a few minutes after that, I was once again impressed with Google’s software and user experience prowess.

Screenshot 1 - Default View

Now that the tracking code has been active on the site for a few weeks, there is enough data to take a good look at most of the features Google Analytics offers. Screenshot 1 shows the default view when you load the application, the “Executive Overview” Dashboard view. This screenshot shows the overall page layout for the reports within Google Analytics, with collapsible report links on the left, a date picking calendar below that, and the reports on the right, with help text below them.

Screenshot 2
Three Dashboard Views

Screenshot 2 shows the three pre-configured dashboard “views”, which seem meant to be specific to the role of the person using Analytics. Ok, maybe I’m getting ahead of myself here. I should mention that Analytics supports the ability to monitor multiple web sites (you set up “Website Profiles” to define them), and that a user with administrator privileges can set up additional users who can access specific profiles only, or all profiles. You can set up more than one profile for the same web site, each with its own set of available reports, which is how you restrict users to seeing only the reports they should be seeing. There is a lot of behind-the-scenes power in Analytics, with features to help small to mid-sized businesses make the most of their online efforts.

Fully expanded Nav bar,
showing all available reports

Now, I am not going to sit here and bore you with a frame-by-frame description of every feature of Google Analytics. I took a lot of screenshots which pretty much speak for themselves, so look them over and you’ll get a sense of the kinds of data this program gives you access to. However, I will mention a few things that are not apparent in these static pics.

Visitor Location Map - Austin has the biggest marker!

First: Browser Compatibility. My own experience is that you pretty much need to be using IE for Google Analytics to work properly. The pages will load in Opera, but all charts are empty, no data gets loaded into them. Firefox support is better (as you can see in Screenshot 1) but other pages do not work, such as the site overlay reports. I switched to IE for the rest of the screenshots.

Site overlay - Each link has metrics data
(very small bars for my site, small data sample size still)

Second: Visitor Counts. Google Analytics gathers visitor information by running JavaScript in the user’s browser when they load a page (client-side method). Although this allows for capturing a wealth of data (such as length of visit, connection speed, and screen resolution) that would not otherwise be available, this collection method inherently leads to lower visitor counts than a server-side method, such as my own hit and click counters. This is because JavaScript must be running in the user’s browser! There are two sets of visitors that are not counted in Google Analytics: automated spiders, or robots, that don’t use a browser at all, and users who switch off JavaScript while they surf.

Visitor Source Pie,
with two slices highlighted

Whether the exclusion of spiders from your Analytics data is a good thing or not depends on your need for that information, and whether you can get it some other way. Of course it’s there in your server log files, but going through them is pretty tedious. In my case, those visits show up in my hit counter data report, and I can trace a particular visit or set of visits to a specific search engine, or I can deduce I’ve been visited by a badbot based on the origin, network used, or pattern of requests. Knowing that I get regular visits from Google, Yahoo, Bloglines, Pluck, IceRocket, and other legitimate search and aggregation sites lets me know my latest content is available for others to find.

An example of a powerful
cross-segment report. Here,
we see the breakdown of
screen resolutions for Firefox users.

You may be wondering how many people really go around the internet with their JavaScript turned off. I didn’t think there were too many of them, but early on I started getting complaints that the blog page layout was all messed up for some folk. It took a little while and exchanges with several complainants for me to realize the problem was they had JavaScript turned off, and since I was using JS code to position the sidebar to the right side of the page, of course it looked terrible for them. I’ve since modified my templates so they don’t rely on JavaScript to position the sidebar, but this experience demonstrated there are a fair percentage of visitors who will not be counted by a client-side data capture method. This is not that big a deal for me, because I am not using Google Analytics for total traffic counts, and I am assuming this exclusion will not skew the report percentages too much. Also, since it’s impossible to buy pixels without JavaScript running, these visitors wouldn’t be in any of the e-commerce related reports anyway.

Visitor Location Pie
with pulled out slice

Finally: Impressions and Opinions. The default date range is the past 7 days when you first load Google Analytics. This range is easy to change via the calendars to the lower left. You can quickly select a standard week, month, or day; a column of days (for example, all Tuesdays in the month); or you can specify an ad-hoc, custom range via the “Enter Range” icon. When you click that icon, two calendars expand into being next to it, where you merely click the start date in one calendar, and the end date in the second one. When you click “Apply Range”, the two new calendars collapse away. Any time you change the date range, the currently displayed report or dashboard is automatically redrawn for the new date(s). From that point on, all reports will use the newly selected date range, unless you change it again. This immediate and “sticky” response to your changes is intuitive and makes it easy to get the report you want. The tool does its job quickly and efficiently, so you can too.

Date Range calendars expand
and collapse on the page - nice

Notice I said the date range calendars “expand into being” and “collapse away”. They do not merely pop into existence all at once, or even more ghastly, open in a new browser window. No, they very pleasantly expand from and collapse into nothingness on the page, a very nice touch. As a web developer, I can appreciate the extra work that goes into implementing such niceties, and although it is arguably just eye candy, little touches like this go a long way towards enhancing the user experience. Likewise, all charts draw themselves over a very short time interval (line graphs trace a path across the grid, and bar graph columns grow from their base to their final length), another nice touch. Ok, pie charts do pop into being all at once, but you can pull out a slice of the pie by clicking on an item in the legend.

Some of the search queries
that led someone to AustinMash!

A couple of gripes: The dashboard and certain reports lack one-click drilldown capability, and the login process could be better integrated with other Google services. When you first load Google Analytics, for example, you may see you have 25% of your visitors being referred to your site by “other”. It should be possible, right there, to click on that pie slice and see the data for “other”. As it is, you have to go to the detailed report (Marketing Optimization/Visitor Segment Performance/Referring Source), and set the display list size to something greater than the default 10 lines. A drilldown click from the dashboard would have been much nicer. Also, Google does not have a single login and authentication mechanism for its various services. Maybe this is by design, but it seems inconsistent with the company’s otherwise user-centric implementations. Perhaps it is a consequence of trying too much, too fast, as various divisions race to provide new applications on their own. One would hope Google will make the effort to tie everything together, from a user account perspective, at some point in the future.

Most visited content, with bounce and exit data

On a happier note, a technically interesting aspect of the reports is the seamless way flash animations are integrated into the page. All the pie, line, and bar charts, plus the geo maps, are Macromedia Flash mini-applets, while the textual data, navigation, and help text are just that, regular HTML text on the page. The calendars are HTML as well, yet all the pieces are interact with each other and are integrated into a well-honed, solidly built web app machine. I like that. Actually, as far as I can tell, this entire application, with all the reports and various date and range picking controls, are really all on the same HTML page. All that happens when you click on a report is that different chart mini-apps get loaded and fed the proper data in the report area of the page, without the whole page refreshing in the browser. Very slick, and in my opinion, the way all web apps should work from now on. Yes, I know Google Maps and Google Finance, as well as numerous other “Web 2.0″ sites (like Flickr) use the same approach. Google just seems to have mastered the art and they are not shy about showing it off. Kudos!

You can compare data from
two time ranges, to measure the
effects of a marketing push, for example

But wait, there’s more! Not only does Google dress to impress, it is also a well behaved, soft-spoken host, which only serves to enhance its popularity in the increasingly crowded internet party scene. After exploring the available reports, it becomes apparent why Google spent the resources to buy the company that developed the core technology, and to enhance and distribute this remarkable tool for free. It’s all there in the “Content Optimization” and “E-Commerce Analysis” sections. As a Google Analytics user, you have the ability to track the effectiveness of various AdWords campaigns, and to analyze the effectiveness of your site in converting visitors into revenue-generating sales. You can define “goals” (such as showing a customer the payment receipt, successful transaction page) and “funnels”, which are paths a visitor would take through your site to reach a “goal”. Then, the reports help you understand how and why visitor landings translate into income. This helps you make more money, some of which you will then spend in more AdWords advertising (with a greater knowledge of what works and what doesn’t), which of course is good for Google.

Another cross-segment report
Where are my Google visitors
coming from?

It seems clear Analytics is one way Google hopes to address its primary weakness, which, as numerous analysts have pointed out, is that despite all the cool technology it produces, Google has very little ability to “lock in” users. In one sense, Google is like the genius kid who enters a hyper-competitive graduate program way too young. Because his peers, such as Microsoft and Yahoo, and Amazon and eBay to a lesser extent, are entrenched and have a built-in ability to keep its users coming back, Google has to be that much better at everything it does so that people will prefer to hang out with it instead of the older kids. So far it has been able to do this, but at the cost of running full speed everywhere it goes, trying to be Mr. popular in all circles to make sure its parties are always well attended. Wouldn’t it be nice to be able to relax a little and not worry about everyone running out the door when the cool new frat house throws its first party? Or worry that folks will get tired of the music and beer and just drift back home to Yahoo and MSN? You bet it would, and if Google’s customers come to rely on Analytics to manage their business, there is much less chance they will eventually migrate away from AdWords or any other revenue-producing service that may be offered – such as the new Google Checkout service.

Some reports help you visualize
how visitors navigate through your site

So, even though Analytics can be considered strategically important to Google, and the AdWords integration is what makes it so, the user experience was not compromised to put these features front and center. Much to Google’s credit, the default views and reports were chosen based on what the users would find most useful, not on what would further the company’s goals. The genius kid doesn’t go around shouting how great it is, it merely does what it does very well, confident that it will pay off in the long run.

I believe that it will – how many Web 1.0 startups when nowhere because they neglected to put the user first (RealNetworks, for one, comes to mind)? Google absolutely needs tools like Analytics to become ingrained in its customer’s business processes, but it seems to understand that providing super-easy, hyper-helpful user experiences is the way to go about it. Despite the muscle under the outfit, Google’s charm and graciousness exudes a confident and understated vibe. This is my kind of party!

Most of my Texas visitors
are from Austin!

Some notes on the data in the screenshots: These of course are actual reports generated from real data about AustinMash!. The spike in traffic on June 15th was due to my blog posting on the recent ROT Biker Rally here in Austin. Since Google Analytics has just started gathering data, most of the visitors are “new” to it and not “returning”. You will note there were some visitors referred to my site from MySpace, I will be talking about my experimentation with that site in a future posting.

Finally, I am gratified that the bulk of my visitors are from Austin. While it’s great to see people in Russia or Saudi Arabia are checking out AustinMash! (and because the data gathering is client-side, I can be fairly sure these are real people and not robots), it has always been my intention to make this site primarily for Austinites. I am very pleased to see my promotional efforts are on-target, that I am doing the right things as far as reaching my intended audience. Very cool indeed.

Most reports include the ability to further graph a metric over time. Here, we see the time distribution of the 23 visitors using Dialup access during this report period.

The browsers people use, and their operating systems. The purple up-arrow icons create new floating graphs with Data-Over-Time, To-Date Lifetime, or Cross-Segment reports. Very powerful stuff.

Got my Flash-based video to play, hurray!

My last post detailed the frustration I experienced in trying to add a simple movie clip to the web site. Most of that frustration was due to the time I spent trying to get Macromedia’s new flash-based movie technology working well enough that other people could see the video, and not just me. Well, I finally got it working, but I’m not sure how.

As I was writing that last post, I tried again to generate a Flash movie that would play on computers that did not have Macromedia Studio 8 installed. At the end of the day it still didn’t work, and I was noticing weird behavior, in that the final SWF file would play on my laptop when it was in the folder where it was “published” to from Flash 8, but not after I copied it to another folder on the same drive!

There was another clue, however, that I did not pay much attention to at first: the file size of the Flash FLA document was considerably smaller than the previous versions of the document I had created a few days ago, even though they all had only the one movie FLV file in them. It seemed like the FLA document no longer included the actual movie, so it must be referencing it externally, which would explain why the movie would not play when it was in a different folder. Ok, the file sizes don’t bear that out: 376 Kb vs 208 Kb for the FLAs does not equal a 2.3 Mb FLV file.

Whatever the case, to test this theory I copied the FLV file to the same folder I where I was putting the SWF file. Lo and behold, it now played! Was is even more puzzling is that both versions of the SWF file, with an internal and external movie reference, are the same size at 35 Kb.

I uploaded all the files to the web site, created a new test page, and verified that it played over the internet on the other computers in the house. I tried it at work, and it played there too. It was fixed.

I suppose it’s possible the FVL movie file was always externally referenced by the SWF, so that the 35 Kb size is only for the player. But remember, this movie played on my laptop from the web site over the internet, before I had uploaded the FLV file. It also played on my local (laptop) version of the web site, again without the FLV file in the folder that had the SWF file. So, how did that work, and why did it stop working, forcing me to upload the FLV? Undoubtedly I changed a setting somewhere when I was tinkering with it trying to make it work. I wish I knew which one it was.

One difference is that for the first attempts, I used Windows Explorer to drag the FLV file from where it was created to the Flash 8 application window. For the last version, I used the File/Import menu to import the video. However, both of these methods activated the same Video Import Wizard, so I don’t think that made a difference.

Ah! I just figured it out! In the non-working flash document, the “contentPath” for the imported video was a full path and file name pointing to another directory, while in the working version, the same parameter just listed the file name! So, the FLV movie was externally referenced, and it needs to be a relative path from the SWF file, preferably in the same folder, otherwise it won’t find it when you upload it to a server. Man, I wish Macromedia would have been a little clearer on that. I suppose in the end this was a bonehead rookie mistake on my part, since I didn’t even find anyone else with this problem in the forums. This also explains why the movie played over the internet on my laptop, because it was looking for and finding the content on my hard drive! Live and learn, I guess.

Anyway, the low-footprint, Flash-based version of the video is now available and should work for everyone who has Flash Player 8 or higher. I just hope I haven’t overhyped this short clip by talking about it so much. It’s really just a small sample of the goings-on at Eeyore’s Birthday this year, just something I wanted to share. Enjoy!

Why is video so frustrating?

All I wanted to do was share with the world a small segment of my life, one I had captured on video. The idea seemed simple: upload the movie file to the web site, add a link to it, and let people enjoy. The video was a short recording of the music and dancing at a drum circle at Eeyore’s birthday a few weeks ago (see this blog entry). Unfortunately, even though the idea seemed simple, the execution was anything but.

In fact, it was confusingly complicated and surprisingly difficult. At this stage in the digital evolution of media, I had expected a seamless experience. Why does video have to be such a pain?

Particularly frustrating was that the process at first seemed to go very smoothly and painlessly. The video was up on the site (I decided to make a dedicated page for it so I could track viewing counts) and I moved on to other things, only to find out later at least some people couldn’t see it.

Recording the scene was very simple. My digital camera, an HP Photosmart R817, has a button next to the main shutter button. You press once to start recording, and again to stop. This was my first time using this feature, and I didn’t know how much memory was going to get chewed up with the video, and since I wanted to leave room for more pictures I didn’t let it record for too long. I should not have worried. When I started recording, the counter said 175 shots left, and when I stopped after about 20 seconds, it said 165 shots left. No problemo, and now I wish I had captured a longer clip.

Once I got home and transfered the mpg file from the camera to my laptop and opened it in Media Player, I was shocked that it had recorded audio as well, and it sounded fairly good! This was great, because at the same time, I had used my cell phone’s note recorder feature to capture the audio just in case. As it turned out, the quality of that recording was considerably crappy. Hmmm, a visual tool recorded better audio than a voice-centric device - go figure. The only problem with the video was that the file was 15 Mb huge - kinda fat for downloading.

Quite coincidentally, I then happened to come across a review of Macromedia’s Studio 8, which includes Flash 8, and it said the new version could now convert movie files into Flash-based movies at high quality. Hey that’s cool, I have Studio 8! I looked in my programs menu, and sure enough, there was the “Macromedia Flash 8 Video Encoder”.

As a first pass to try out the technology, I used the default settings all the way to creating an HTML output page with Flash 8. The process involved encoding the original MPG file into a Flash Video File (FLV), which resulted in a 6.2 Mb file. Then, that file is imported into a new Flash Document, a player skin is chosen from the templates, and the document is “published” to an HTML output, which includes the movie SWF file, the skin SWF, and the HTML container page.

The resulting SWF Flash-based movie was only 35 Kb, and it looked just as good as the original footage - pretty neat stuff. The only problem was that the bottom of the scene was cut off, so I had to go back into the publish settings and set the Dimensions to “Match Movie”. It’s odd that this was not the default setting.

The last step (or so I thought) was to copy and paste the Macromedia-generated HTML code into a new blank page that integrated into the web site, upload everything, and make the links to it in my blog post. It all worked great on my laptop, and I was happy with the result. Until, of course, I got reports that the movie was not viewable over the internet.

Since I was at work at the time I first heard about it, I tried it from there, and sure enough, the video did not show in the browser, all I got was a large blank space on the page - what the heck? This was several days after I had posted the links, and my hit counter showed the movie page was being viewed. So, did this mean people wanted to see the video, but the page was blank for them? That was pretty bad. On top of that, one of the drags about having a day job is that I can’t just fix things on the web site when they crop up, I have to wait until I get home that evening. The thought of people getting a page that did not work gnawed at me the rest of the day. Bummer.

When I got home, I noticed that the video didn’t work on my laptop when clicking on the link in the blog post, whereas before it had. After trying a number of different things, I discovered that if the domain of the HTML page and the domain of the SWF page do not match, the movie does not show. I assume this is some kind of security feature in Flash Player. What was frustrating, though, is that only the sub-domains were different, and the security was apparently not smart enough to handle that. The links in the blog post pointed to http://austinmash.com/drums.php. Within that file, the path to the movie clip was http://www.austinmash.com/Images/drums2.swf. Normally, whether you include the “www.” in the address or not, you get to the same place. But no, Flash didn’t see it that way, no sirreee, no how no way!

So I added the “www.” to the blog links, added a note on the movie page itself so the user makes sure the address is correct, and made a comment on the blog posting apologizing for the snafu. Then I went to bed. The next day at work, I tried it again, and the movie was still blank! What the heck? Again, I had to wait until I got home before I could do anything about it.

This time there were no easy answers. I tried a number of different settings for both the mpg-to-flv and flv-to-swf processes, and while they all played on my laptop, they did not on the other computers in the house that did not have Studio 8 installed (of course, they did have Flash Player 8. Otherwise, the page shows a link to upgrade the player, and not a large blank area). Online research was not any help, I did not find anyone else complaining of this problem, much less a solution to it. So, after spending way too much time trying to get the flash-based movie to work, I gave up and decided to just show “regular” video and not worry about the bandwidth it might eat up.

I was still unwilling to post a 15 Mb clip, though, so I looked into ways to compress that, and I wanted to give users the option of using either Quicktime or Windows Media player. Theoretically, both players can show mpg files, but the reality was that they both did a poor job with that format, so I had to find ways to convert the movie to their respective native formats, WMV for Windows and MOV for Quicktime. That’s when things got really hairy. First, I had to find an app that can do the conversion, then figure out the best settings for the video and audio codecs, and then write appropriate HTML code for the players. Oh what fun.

I spent several hours downloading and trying a few video conversion apps. Most were hard to use and assumed a level of codec familiarity I did not have. Why does video have to be so complicated? I finally ended up using a program called ImTOO MGEG Encoder , which not only handled all the relevant formats, it is configured with default profiles for each, taking a lot of the guesswork out of the process. It produced 4.1 and 4.8 Mb files for the WMV and MOV formats, respectively, with little or no loss of quality. Cool.

Then instead of making two pages, each with one of the two players, I made one page with Javascript to write the appropriate code for the player the user clicked as their preference. This dynamically adds the players to the page, which gets around IE’s new behavior that forces users to click on a control to use it, which was implemented as a response to Microsoft getting sued for seamlessly integrating controls in web pages, because someone else thought of it first and patented the idea. I also removed the blog comment about the “www.” in the address for the movie page, as that was obsolete now.

Anyway, what should have been a straightforward video clip posting turned out to be a mutli-day effort, with a less than optimal solution. I’ll continue to play with Flash to see if I can make it work, but this technology seems to be rather flaky still. While I was writing this post, I went back and tried again. I found a player parameter called “allowScriptAccess”, which was set to “sameDomain” by the HTML generator. I changed it to “always”, as per the documentation, but that didn’t help. Also, I published to a new SWF file, which played on it’s own while in the directory it was created in, but not after I copied it to my local web site directory! It’s the same file, just it won’t play from a different directory like the previous versions did - that’s flaky behavior, folks. Why is video so frustrating?

The final result is here.

AustinMania! is now a blog, too, and you can post, too

The previous post describes my investigation into Ruby on Rails, as a framework I could use to rapidly build out the AustinMania! community portal. Alas, I was once again stymied by my web host, and for now, I cannot run RoR applications on this web site. This may change with further study, but luckily, I do not need to wait for that to significantly enhance the interactiveness of the site.

As I was pondering my next move, which included considering if I should switch web hosts, a sudden realization struck me – I already had all I needed to implement the beginnings of a user-generated community – a WordPress blog! All I had to do was to make a new, separate database, re-install WordPress with a new blog name, and presto, a new blog was created on the site! The main difference in functionality vs my regular blog is that when a new user registers, they are by default granted “contributor” level access, which means they can create new posts themselves (subject to approval by me before they are published). I also opened up commenting privileges so anyone can comment, and they appear right away without moderation. I can always go back and remove inappropriate comments if I need to, and since the initial goal is to encourage use, this should be ok for now.

I made a few changes to the sidebar from the one this blog is running to better fit the community aspect of AustinMania!, such as listing all the authors and removing the blogroll. Also, I tweaked the entire user registration and login process to “hide” the fact AustinMania! is really a blog, it’s supposed to look like just a community website. I added helpful instructions to the e-mail that sends the initial password, and to some of the registration pages, and removed certain pages or sections from the admin console to simplify it for people who have never blogged before. For example, the dashboard seemed fairly superfluous, so it’s gone. I will probably continue to tweak the admin pages over time so they better fit the theme of AustinMania!

The best part of this is that it is up and running, anyone can contribute, and I can continue to tweak it and add features. I still intend to bring to life my “grand vision” of what AustinMania! can be, but at this point I am not sure if it will evolve from this WordPress platform, or if I will build an entirely new RoR application. Perhaps in the end it will be a hybrid. Whatever the case, I am glad I was able to implement something that works today, and I didn’t let the bigger ideas get in the way of progress. I hate it when that happens, don’t you?

Ruby on Rails stalls pulling out of the station

Sometimes the grand vision can get in the way. It has always been my intent to make the AustinMania! section of this site the part of this endeavor that “captures the essence of Austin online” and to let the community shape it, rather than create the whole thing myself. As I went about building the other sections of the site, certain ideas about how I could achieve this vague goal floated around in my head, pretty much in the background.

These flights of fancy, regarding features and usability flow, soared high and far, but they always came back to earth crashing amid the realization of the amount of work required to pull them off. Now work in itself is not a bad thing, the problem is the implementation delay when the task at hand is extensive, and hard to implement piecemeal. So, as a stopgap, a static page was created for AustinMania! that asked visitors to send in contributions via e-mail. This was hardly ideal, but at least it was real.

Meanwhile, my ideas for AustinMania! went in a direction that I started to get excited about, something fairly unique (as far as I know) but still based on existing and recognizable trends in cyberspace. It was unlimited in depth, supported and expanded on the original business model, and would be pretty cool to boot – if I could ever get it done. Again, the same brick wall of feasibility sprang up and cut the fancy flights short.

By and by, at some point I decided to see what all the fuss over Ruby on Rails (RoR) was about. I watched the famous “Creating a weblog in 15 minutes” video , and was impressed by the apparent ease of use, robustness, flexibility, and speed with which the application was created. The part when the presenter merely adds a date/time field to the underlying database, and then reloads a page in the browser that is running the application, and lo and behold, the UI automagically produces a date entry form field for user interaction shows how well all the pieces (model, viewer, and controller) are wired together, automatically, by default, with no real effort on the part of the developer. This is a serious time saver, here.

Ok, that’s all well and dandy, but I like to have ultimate control over how pages look and function, so anytime I see auto-generated code, I start to loose interest. Surely, the developers of the framework do not share my vision of how things should work, and I definitely do not want to be constrained by their views, so no thanks. But wait – this video was not yet over, they were only 7 minutes into it (and already the app was functional). Sure enough, the next thing that was done in the video was a tweak to the templates used to generate the pages, and I saw that they were made with simple, clean HTML code with a few extra tags thrown in. With a few keystrokes, the default tabular layout was replaced with a list-based layout, using regular HTML, and the UI instantly re-rendered with the new look and functionality. Wow – total control over the output, and auto-wiring to the database, and simple controller action definitions. And, um, this is open source, and free for me to use? Yep, I was SOLD!

So I downloaded the necessary pieces and very soon, I had a rudimentary working application. I also added a “UserEngine”, which I found in the rails engines repository , and which also means that within about 5 minutes after downloading it, I had the major pieces allowing new user signup, existing user login, and account maintenance. Even better, one line of code at the top of every password protected page (or just one line in the header template) implemented this role-based access control mechanism for that page. It is no small wonder there’s a fair bit of hype surrounding this framework, the productivity gains are impressive!

But then came the final test – would this actually run on my web site, on my web host’s servers? Well, there’s the snag. Most of the articles and documentation I’ve seen that talk about deployment assume your RoR production app will be hosted on your own server, over which you have total control. There are instructions for how to deploy to the maybe 2 or 3 web hosting companies that have Ruby installed and supported, and, supposedly you can “make it work” on other Apache servers. But the documentation on that is a work in progress, and assumes the ability to edit the .htaccess and other server configuration files. As regular readers of this blog know, I have no such access on my web host. So at first glance, it would seem I cannot run RoR apps on my web site.

But then again, as is also known to those who follow my story, I created a workaround for my lack of appropriate server access by using PHP header redirects. This was done so that this blog could use custom Permalinks for individual postings. Would this technique also work with RoR? Well - no, it didn’t work, at least not right away. It’s possible I do not yet understand well enough how RoR works, so this may still be feasible, but it will take more experimenting.

So I’ll continue to explore RoR and ponder my deployment options. In the meantime, I’ve come up with a relatively simple way to enhance AustinMania!, still far short of the big vision, but at least a lot more interesting than a static page. Check out the next post for details…