Although this setup worked great for its intended purpose, it left a lot of unanswered questions related to what users were doing on the site. For example, I started to see, in the report page, a large number of page views from the same person. I mean, page views in the 60-90s in the same day, and again the next day, and the next. Since there are only about 40 pages in the whole site, this was pretty weird. I can appreciate someone being so mesmerized by my site they keep re-reading everything, but come-on, over 200 page views in 3 days? I had to go dig into the server log files to see that this user was not going through all the content at all, all requests were the same, for the blog home page! The logs also showed all the requests were being referred to from BlogSlides . Aha! I had recently added my blog to their index of blogs, and their unique feature is that they automatically cycle through a series of blogs in the same browser window, hence the name BlogSlides. Sure enough, the logs showed this visitor loaded the main blog page at regular intervals all day long. Ok, so this minor mystery turned out to be someone who left his BlogSlides window running, and by doing so created a bunch of bogus requests for all the blogs in his slide show, not just mine, providing another example of the unintended consequences of someone trying something new. Anyway, I found my answer through tedious scanning of my log files, not through my automated visitor counters. This would have been a lot easier with dedicated Web Site Analysis software.

So, I knew my hit and click counters were not good at providing the kind of deep analysis you can get from web analytics software, they were never intended to be, and creating such a system was never a reason for AustinMash!’s existence. All along, I’ve thought that if detailed visitor stats ever became important to me, I would have to shell out several hundred dollars for a decent and robust solution, something like WebTrends .

Then, a little while ago, an interesting e-mail appeared in my inbox – it was an invitation from Google to try out their new Analytics solution, which was, of course, free! Apparently I had asked for an invitation some time ago, probably when I set up my AdSense advertising account, and then forgot all about it. So, here was Google, one of the most impressive software developers of today (Google Maps, Google Finance, Google Spreadsheet, need I go on?) inviting me to try their latest creation. Guess how long it took me to add their tracking code to my pages? Not very, I’ll tell you that right now.

The hardest part in adding their code (about 4 lines of JavaScript) was in deciding where it should go. In my case, since I have made the commitment to have the Skyline appear on all publicly accessible pages, it made sense to put the Google code next to the code that generates the Skyline Ads area map. Although Google says it may take a day, within a few hours of uploading the new code to the site my personal Analytics page was showing visitor data, and a few minutes after that, I was once again impressed with Google’s software and user experience prowess.

Screenshot 1 - Default View

Now that the tracking code has been active on the site for a few weeks, there is enough data to take a good look at most of the features Google Analytics offers. Screenshot 1 shows the default view when you load the application, the “Executive Overview” Dashboard view. This screenshot shows the overall page layout for the reports within Google Analytics, with collapsible report links on the left, a date picking calendar below that, and the reports on the right, with help text below them.

Screenshot 2
Three Dashboard Views

Screenshot 2 shows the three pre-configured dashboard “views”, which seem meant to be specific to the role of the person using Analytics. Ok, maybe I’m getting ahead of myself here. I should mention that Analytics supports the ability to monitor multiple web sites (you set up “Website Profiles” to define them), and that a user with administrator privileges can set up additional users who can access specific profiles only, or all profiles. You can set up more than one profile for the same web site, each with its own set of available reports, which is how you restrict users to seeing only the reports they should be seeing. There is a lot of behind-the-scenes power in Analytics, with features to help small to mid-sized businesses make the most of their online efforts.

Fully expanded Nav bar,
showing all available reports

Now, I am not going to sit here and bore you with a frame-by-frame description of every feature of Google Analytics. I took a lot of screenshots which pretty much speak for themselves, so look them over and you’ll get a sense of the kinds of data this program gives you access to. However, I will mention a few things that are not apparent in these static pics.

Visitor Location Map - Austin has the biggest marker!

First: Browser Compatibility. My own experience is that you pretty much need to be using IE for Google Analytics to work properly. The pages will load in Opera, but all charts are empty, no data gets loaded into them. Firefox support is better (as you can see in Screenshot 1) but other pages do not work, such as the site overlay reports. I switched to IE for the rest of the screenshots.

Site overlay - Each link has metrics data
(very small bars for my site, small data sample size still)

Second: Visitor Counts. Google Analytics gathers visitor information by running JavaScript in the user’s browser when they load a page (client-side method). Although this allows for capturing a wealth of data (such as length of visit, connection speed, and screen resolution) that would not otherwise be available, this collection method inherently leads to lower visitor counts than a server-side method, such as my own hit and click counters. This is because JavaScript must be running in the user’s browser! There are two sets of visitors that are not counted in Google Analytics: automated spiders, or robots, that don’t use a browser at all, and users who switch off JavaScript while they surf.

Visitor Source Pie,
with two slices highlighted

Whether the exclusion of spiders from your Analytics data is a good thing or not depends on your need for that information, and whether you can get it some other way. Of course it’s there in your server log files, but going through them is pretty tedious. In my case, those visits show up in my hit counter data report, and I can trace a particular visit or set of visits to a specific search engine, or I can deduce I’ve been visited by a badbot based on the origin, network used, or pattern of requests. Knowing that I get regular visits from Google, Yahoo, Bloglines, Pluck, IceRocket, and other legitimate search and aggregation sites lets me know my latest content is available for others to find.

An example of a powerful
cross-segment report. Here,
we see the breakdown of
screen resolutions for Firefox users.

You may be wondering how many people really go around the internet with their JavaScript turned off. I didn’t think there were too many of them, but early on I started getting complaints that the blog page layout was all messed up for some folk. It took a little while and exchanges with several complainants for me to realize the problem was they had JavaScript turned off, and since I was using JS code to position the sidebar to the right side of the page, of course it looked terrible for them. I’ve since modified my templates so they don’t rely on JavaScript to position the sidebar, but this experience demonstrated there are a fair percentage of visitors who will not be counted by a client-side data capture method. This is not that big a deal for me, because I am not using Google Analytics for total traffic counts, and I am assuming this exclusion will not skew the report percentages too much. Also, since it’s impossible to buy pixels without JavaScript running, these visitors wouldn’t be in any of the e-commerce related reports anyway.

Visitor Location Pie
with pulled out slice

Finally: Impressions and Opinions. The default date range is the past 7 days when you first load Google Analytics. This range is easy to change via the calendars to the lower left. You can quickly select a standard week, month, or day; a column of days (for example, all Tuesdays in the month); or you can specify an ad-hoc, custom range via the “Enter Range” icon. When you click that icon, two calendars expand into being next to it, where you merely click the start date in one calendar, and the end date in the second one. When you click “Apply Range”, the two new calendars collapse away. Any time you change the date range, the currently displayed report or dashboard is automatically redrawn for the new date(s). From that point on, all reports will use the newly selected date range, unless you change it again. This immediate and “sticky” response to your changes is intuitive and makes it easy to get the report you want. The tool does its job quickly and efficiently, so you can too.

Date Range calendars expand
and collapse on the page - nice

Notice I said the date range calendars “expand into being” and “collapse away”. They do not merely pop into existence all at once, or even more ghastly, open in a new browser window. No, they very pleasantly expand from and collapse into nothingness on the page, a very nice touch. As a web developer, I can appreciate the extra work that goes into implementing such niceties, and although it is arguably just eye candy, little touches like this go a long way towards enhancing the user experience. Likewise, all charts draw themselves over a very short time interval (line graphs trace a path across the grid, and bar graph columns grow from their base to their final length), another nice touch. Ok, pie charts do pop into being all at once, but you can pull out a slice of the pie by clicking on an item in the legend.

Some of the search queries
that led someone to AustinMash!

A couple of gripes: The dashboard and certain reports lack one-click drilldown capability, and the login process could be better integrated with other Google services. When you first load Google Analytics, for example, you may see you have 25% of your visitors being referred to your site by “other”. It should be possible, right there, to click on that pie slice and see the data for “other”. As it is, you have to go to the detailed report (Marketing Optimization/Visitor Segment Performance/Referring Source), and set the display list size to something greater than the default 10 lines. A drilldown click from the dashboard would have been much nicer. Also, Google does not have a single login and authentication mechanism for its various services. Maybe this is by design, but it seems inconsistent with the company’s otherwise user-centric implementations. Perhaps it is a consequence of trying too much, too fast, as various divisions race to provide new applications on their own. One would hope Google will make the effort to tie everything together, from a user account perspective, at some point in the future.

Most visited content, with bounce and exit data

On a happier note, a technically interesting aspect of the reports is the seamless way flash animations are integrated into the page. All the pie, line, and bar charts, plus the geo maps, are Macromedia Flash mini-applets, while the textual data, navigation, and help text are just that, regular HTML text on the page. The calendars are HTML as well, yet all the pieces are interact with each other and are integrated into a well-honed, solidly built web app machine. I like that. Actually, as far as I can tell, this entire application, with all the reports and various date and range picking controls, are really all on the same HTML page. All that happens when you click on a report is that different chart mini-apps get loaded and fed the proper data in the report area of the page, without the whole page refreshing in the browser. Very slick, and in my opinion, the way all web apps should work from now on. Yes, I know Google Maps and Google Finance, as well as numerous other “Web 2.0″ sites (like Flickr) use the same approach. Google just seems to have mastered the art and they are not shy about showing it off. Kudos!

You can compare data from
two time ranges, to measure the
effects of a marketing push, for example

But wait, there’s more! Not only does Google dress to impress, it is also a well behaved, soft-spoken host, which only serves to enhance its popularity in the increasingly crowded internet party scene. After exploring the available reports, it becomes apparent why Google spent the resources to buy the company that developed the core technology, and to enhance and distribute this remarkable tool for free. It’s all there in the “Content Optimization” and “E-Commerce Analysis” sections. As a Google Analytics user, you have the ability to track the effectiveness of various AdWords campaigns, and to analyze the effectiveness of your site in converting visitors into revenue-generating sales. You can define “goals” (such as showing a customer the payment receipt, successful transaction page) and “funnels”, which are paths a visitor would take through your site to reach a “goal”. Then, the reports help you understand how and why visitor landings translate into income. This helps you make more money, some of which you will then spend in more AdWords advertising (with a greater knowledge of what works and what doesn’t), which of course is good for Google.

Another cross-segment report
Where are my Google visitors
coming from?

It seems clear Analytics is one way Google hopes to address its primary weakness, which, as numerous analysts have pointed out, is that despite all the cool technology it produces, Google has very little ability to “lock in” users. In one sense, Google is like the genius kid who enters a hyper-competitive graduate program way too young. Because his peers, such as Microsoft and Yahoo, and Amazon and eBay to a lesser extent, are entrenched and have a built-in ability to keep its users coming back, Google has to be that much better at everything it does so that people will prefer to hang out with it instead of the older kids. So far it has been able to do this, but at the cost of running full speed everywhere it goes, trying to be Mr. popular in all circles to make sure its parties are always well attended. Wouldn’t it be nice to be able to relax a little and not worry about everyone running out the door when the cool new frat house throws its first party? Or worry that folks will get tired of the music and beer and just drift back home to Yahoo and MSN? You bet it would, and if Google’s customers come to rely on Analytics to manage their business, there is much less chance they will eventually migrate away from AdWords or any other revenue-producing service that may be offered – such as the new Google Checkout service.

Some reports help you visualize
how visitors navigate through your site

So, even though Analytics can be considered strategically important to Google, and the AdWords integration is what makes it so, the user experience was not compromised to put these features front and center. Much to Google’s credit, the default views and reports were chosen based on what the users would find most useful, not on what would further the company’s goals. The genius kid doesn’t go around shouting how great it is, it merely does what it does very well, confident that it will pay off in the long run.

I believe that it will – how many Web 1.0 startups when nowhere because they neglected to put the user first (RealNetworks, for one, comes to mind)? Google absolutely needs tools like Analytics to become ingrained in its customer’s business processes, but it seems to understand that providing super-easy, hyper-helpful user experiences is the way to go about it. Despite the muscle under the outfit, Google’s charm and graciousness exudes a confident and understated vibe. This is my kind of party!

Most of my Texas visitors
are from Austin!

Some notes on the data in the screenshots: These of course are actual reports generated from real data about AustinMash!. The spike in traffic on June 15th was due to my blog posting on the recent ROT Biker Rally here in Austin. Since Google Analytics has just started gathering data, most of the visitors are “new” to it and not “returning”. You will note there were some visitors referred to my site from MySpace, I will be talking about my experimentation with that site in a future posting.

Finally, I am gratified that the bulk of my visitors are from Austin. While it’s great to see people in Russia or Saudi Arabia are checking out AustinMash! (and because the data gathering is client-side, I can be fairly sure these are real people and not robots), it has always been my intention to make this site primarily for Austinites. I am very pleased to see my promotional efforts are on-target, that I am doing the right things as far as reaching my intended audience. Very cool indeed.

Most reports include the ability to further graph a metric over time. Here, we see the time distribution of the 23 visitors using Dialup access during this report period.

The browsers people use, and their operating systems. The purple up-arrow icons create new floating graphs with Data-Over-Time, To-Date Lifetime, or Cross-Segment reports. Very powerful stuff.

As I was writing that last post, I tried again to generate a Flash movie that would play on computers that did not have Macromedia Studio 8 installed. At the end of the day it still didn’t work, and I was noticing weird behavior, in that the final SWF file would play on my laptop when it was in the folder where it was “published” to from Flash 8, but not after I copied it to another folder on the same drive!

There was another clue, however, that I did not pay much attention to at first: the file size of the Flash FLA document was considerably smaller than the previous versions of the document I had created a few days ago, even though they all had only the one movie FLV file in them. It seemed like the FLA document no longer included the actual movie, so it must be referencing it externally, which would explain why the movie would not play when it was in a different folder. Ok, the file sizes don’t bear that out: 376 Kb vs 208 Kb for the FLAs does not equal a 2.3 Mb FLV file.

Whatever the case, to test this theory I copied the FLV file to the same folder I where I was putting the SWF file. Lo and behold, it now played! Was is even more puzzling is that both versions of the SWF file, with an internal and external movie reference, are the same size at 35 Kb.

I uploaded all the files to the web site, created a new test page, and verified that it played over the internet on the other computers in the house. I tried it at work, and it played there too. It was fixed.

I suppose it’s possible the FVL movie file was always externally referenced by the SWF, so that the 35 Kb size is only for the player. But remember, this movie played on my laptop from the web site over the internet, before I had uploaded the FLV file. It also played on my local (laptop) version of the web site, again without the FLV file in the folder that had the SWF file. So, how did that work, and why did it stop working, forcing me to upload the FLV? Undoubtedly I changed a setting somewhere when I was tinkering with it trying to make it work. I wish I knew which one it was.

One difference is that for the first attempts, I used Windows Explorer to drag the FLV file from where it was created to the Flash 8 application window. For the last version, I used the File/Import menu to import the video. However, both of these methods activated the same Video Import Wizard, so I don’t think that made a difference.

Ah! I just figured it out! In the non-working flash document, the “contentPath” for the imported video was a full path and file name pointing to another directory, while in the working version, the same parameter just listed the file name! So, the FLV movie was externally referenced, and it needs to be a relative path from the SWF file, preferably in the same folder, otherwise it won’t find it when you upload it to a server. Man, I wish Macromedia would have been a little clearer on that. I suppose in the end this was a bonehead rookie mistake on my part, since I didn’t even find anyone else with this problem in the forums. This also explains why the movie played over the internet on my laptop, because it was looking for and finding the content on my hard drive! Live and learn, I guess.

Anyway, the low-footprint, Flash-based version of the video is now available and should work for everyone who has Flash Player 8 or higher. I just hope I haven’t overhyped this short clip by talking about it so much. It’s really just a small sample of the goings-on at Eeyore’s Birthday this year, just something I wanted to share. Enjoy!

In fact, it was confusingly complicated and surprisingly difficult. At this stage in the digital evolution of media, I had expected a seamless experience. Why does video have to be such a pain?

Particularly frustrating was that the process at first seemed to go very smoothly and painlessly. The video was up on the site (I decided to make a dedicated page for it so I could track viewing counts) and I moved on to other things, only to find out later at least some people couldn’t see it.

Recording the scene was very simple. My digital camera, an HP Photosmart R817, has a button next to the main shutter button. You press once to start recording, and again to stop. This was my first time using this feature, and I didn’t know how much memory was going to get chewed up with the video, and since I wanted to leave room for more pictures I didn’t let it record for too long. I should not have worried. When I started recording, the counter said 175 shots left, and when I stopped after about 20 seconds, it said 165 shots left. No problemo, and now I wish I had captured a longer clip.

Once I got home and transfered the mpg file from the camera to my laptop and opened it in Media Player, I was shocked that it had recorded audio as well, and it sounded fairly good! This was great, because at the same time, I had used my cell phone’s note recorder feature to capture the audio just in case. As it turned out, the quality of that recording was considerably crappy. Hmmm, a visual tool recorded better audio than a voice-centric device - go figure. The only problem with the video was that the file was 15 Mb huge - kinda fat for downloading.

Quite coincidentally, I then happened to come across a review of Macromedia’s Studio 8, which includes Flash 8, and it said the new version could now convert movie files into Flash-based movies at high quality. Hey that’s cool, I have Studio 8! I looked in my programs menu, and sure enough, there was the “Macromedia Flash 8 Video Encoder”.

As a first pass to try out the technology, I used the default settings all the way to creating an HTML output page with Flash 8. The process involved encoding the original MPG file into a Flash Video File (FLV), which resulted in a 6.2 Mb file. Then, that file is imported into a new Flash Document, a player skin is chosen from the templates, and the document is “published” to an HTML output, which includes the movie SWF file, the skin SWF, and the HTML container page.

The resulting SWF Flash-based movie was only 35 Kb, and it looked just as good as the original footage - pretty neat stuff. The only problem was that the bottom of the scene was cut off, so I had to go back into the publish settings and set the Dimensions to “Match Movie”. It’s odd that this was not the default setting.

The last step (or so I thought) was to copy and paste the Macromedia-generated HTML code into a new blank page that integrated into the web site, upload everything, and make the links to it in my blog post. It all worked great on my laptop, and I was happy with the result. Until, of course, I got reports that the movie was not viewable over the internet.

Since I was at work at the time I first heard about it, I tried it from there, and sure enough, the video did not show in the browser, all I got was a large blank space on the page - what the heck? This was several days after I had posted the links, and my hit counter showed the movie page was being viewed. So, did this mean people wanted to see the video, but the page was blank for them? That was pretty bad. On top of that, one of the drags about having a day job is that I can’t just fix things on the web site when they crop up, I have to wait until I get home that evening. The thought of people getting a page that did not work gnawed at me the rest of the day. Bummer.

When I got home, I noticed that the video didn’t work on my laptop when clicking on the link in the blog post, whereas before it had. After trying a number of different things, I discovered that if the domain of the HTML page and the domain of the SWF page do not match, the movie does not show. I assume this is some kind of security feature in Flash Player. What was frustrating, though, is that only the sub-domains were different, and the security was apparently not smart enough to handle that. The links in the blog post pointed to http://austinmash.com/drums.php. Within that file, the path to the movie clip was http://www.austinmash.com/Images/drums2.swf. Normally, whether you include the “www.” in the address or not, you get to the same place. But no, Flash didn’t see it that way, no sirreee, no how no way!

So I added the “www.” to the blog links, added a note on the movie page itself so the user makes sure the address is correct, and made a comment on the blog posting apologizing for the snafu. Then I went to bed. The next day at work, I tried it again, and the movie was still blank! What the heck? Again, I had to wait until I got home before I could do anything about it.

This time there were no easy answers. I tried a number of different settings for both the mpg-to-flv and flv-to-swf processes, and while they all played on my laptop, they did not on the other computers in the house that did not have Studio 8 installed (of course, they did have Flash Player 8. Otherwise, the page shows a link to upgrade the player, and not a large blank area). Online research was not any help, I did not find anyone else complaining of this problem, much less a solution to it. So, after spending way too much time trying to get the flash-based movie to work, I gave up and decided to just show “regular” video and not worry about the bandwidth it might eat up.

I was still unwilling to post a 15 Mb clip, though, so I looked into ways to compress that, and I wanted to give users the option of using either Quicktime or Windows Media player. Theoretically, both players can show mpg files, but the reality was that they both did a poor job with that format, so I had to find ways to convert the movie to their respective native formats, WMV for Windows and MOV for Quicktime. That’s when things got really hairy. First, I had to find an app that can do the conversion, then figure out the best settings for the video and audio codecs, and then write appropriate HTML code for the players. Oh what fun.

I spent several hours downloading and trying a few video conversion apps. Most were hard to use and assumed a level of codec familiarity I did not have. Why does video have to be so complicated? I finally ended up using a program called ImTOO MGEG Encoder , which not only handled all the relevant formats, it is configured with default profiles for each, taking a lot of the guesswork out of the process. It produced 4.1 and 4.8 Mb files for the WMV and MOV formats, respectively, with little or no loss of quality. Cool.

Then instead of making two pages, each with one of the two players, I made one page with Javascript to write the appropriate code for the player the user clicked as their preference. This dynamically adds the players to the page, which gets around IE’s new behavior that forces users to click on a control to use it, which was implemented as a response to Microsoft getting sued for seamlessly integrating controls in web pages, because someone else thought of it first and patented the idea. I also removed the blog comment about the “www.” in the address for the movie page, as that was obsolete now.

Anyway, what should have been a straightforward video clip posting turned out to be a mutli-day effort, with a less than optimal solution. I’ll continue to play with Flash to see if I can make it work, but this technology seems to be rather flaky still. While I was writing this post, I went back and tried again. I found a player parameter called “allowScriptAccess”, which was set to “sameDomain” by the HTML generator. I changed it to “always”, as per the documentation, but that didn’t help. Also, I published to a new SWF file, which played on it’s own while in the directory it was created in, but not after I copied it to my local web site directory! It’s the same file, just it won’t play from a different directory like the previous versions did - that’s flaky behavior, folks. Why is video so frustrating?

The final result is here.

As I was pondering my next move, which included considering if I should switch web hosts, a sudden realization struck me – I already had all I needed to implement the beginnings of a user-generated community – a WordPress blog! All I had to do was to make a new, separate database, re-install WordPress with a new blog name, and presto, a new blog was created on the site! The main difference in functionality vs my regular blog is that when a new user registers, they are by default granted “contributor” level access, which means they can create new posts themselves (subject to approval by me before they are published). I also opened up commenting privileges so anyone can comment, and they appear right away without moderation. I can always go back and remove inappropriate comments if I need to, and since the initial goal is to encourage use, this should be ok for now.

I made a few changes to the sidebar from the one this blog is running to better fit the community aspect of AustinMania!, such as listing all the authors and removing the blogroll. Also, I tweaked the entire user registration and login process to “hide” the fact AustinMania! is really a blog, it’s supposed to look like just a community website. I added helpful instructions to the e-mail that sends the initial password, and to some of the registration pages, and removed certain pages or sections from the admin console to simplify it for people who have never blogged before. For example, the dashboard seemed fairly superfluous, so it’s gone. I will probably continue to tweak the admin pages over time so they better fit the theme of AustinMania!

The best part of this is that it is up and running, anyone can contribute, and I can continue to tweak it and add features. I still intend to bring to life my “grand vision” of what AustinMania! can be, but at this point I am not sure if it will evolve from this WordPress platform, or if I will build an entirely new RoR application. Perhaps in the end it will be a hybrid. Whatever the case, I am glad I was able to implement something that works today, and I didn’t let the bigger ideas get in the way of progress. I hate it when that happens, don’t you?

These flights of fancy, regarding features and usability flow, soared high and far, but they always came back to earth crashing amid the realization of the amount of work required to pull them off. Now work in itself is not a bad thing, the problem is the implementation delay when the task at hand is extensive, and hard to implement piecemeal. So, as a stopgap, a static page was created for AustinMania! that asked visitors to send in contributions via e-mail. This was hardly ideal, but at least it was real.

Meanwhile, my ideas for AustinMania! went in a direction that I started to get excited about, something fairly unique (as far as I know) but still based on existing and recognizable trends in cyberspace. It was unlimited in depth, supported and expanded on the original business model, and would be pretty cool to boot – if I could ever get it done. Again, the same brick wall of feasibility sprang up and cut the fancy flights short.

By and by, at some point I decided to see what all the fuss over Ruby on Rails (RoR) was about. I watched the famous “Creating a weblog in 15 minutes” video , and was impressed by the apparent ease of use, robustness, flexibility, and speed with which the application was created. The part when the presenter merely adds a date/time field to the underlying database, and then reloads a page in the browser that is running the application, and lo and behold, the UI automagically produces a date entry form field for user interaction shows how well all the pieces (model, viewer, and controller) are wired together, automatically, by default, with no real effort on the part of the developer. This is a serious time saver, here.

Ok, that’s all well and dandy, but I like to have ultimate control over how pages look and function, so anytime I see auto-generated code, I start to loose interest. Surely, the developers of the framework do not share my vision of how things should work, and I definitely do not want to be constrained by their views, so no thanks. But wait – this video was not yet over, they were only 7 minutes into it (and already the app was functional). Sure enough, the next thing that was done in the video was a tweak to the templates used to generate the pages, and I saw that they were made with simple, clean HTML code with a few extra tags thrown in. With a few keystrokes, the default tabular layout was replaced with a list-based layout, using regular HTML, and the UI instantly re-rendered with the new look and functionality. Wow – total control over the output, and auto-wiring to the database, and simple controller action definitions. And, um, this is open source, and free for me to use? Yep, I was SOLD!

So I downloaded the necessary pieces and very soon, I had a rudimentary working application. I also added a “UserEngine”, which I found in the rails engines repository , and which also means that within about 5 minutes after downloading it, I had the major pieces allowing new user signup, existing user login, and account maintenance. Even better, one line of code at the top of every password protected page (or just one line in the header template) implemented this role-based access control mechanism for that page. It is no small wonder there’s a fair bit of hype surrounding this framework, the productivity gains are impressive!

But then came the final test – would this actually run on my web site, on my web host’s servers? Well, there’s the snag. Most of the articles and documentation I’ve seen that talk about deployment assume your RoR production app will be hosted on your own server, over which you have total control. There are instructions for how to deploy to the maybe 2 or 3 web hosting companies that have Ruby installed and supported, and, supposedly you can “make it work” on other Apache servers. But the documentation on that is a work in progress, and assumes the ability to edit the .htaccess and other server configuration files. As regular readers of this blog know, I have no such access on my web host. So at first glance, it would seem I cannot run RoR apps on my web site.

But then again, as is also known to those who follow my story, I created a workaround for my lack of appropriate server access by using PHP header redirects. This was done so that this blog could use custom Permalinks for individual postings. Would this technique also work with RoR? Well - no, it didn’t work, at least not right away. It’s possible I do not yet understand well enough how RoR works, so this may still be feasible, but it will take more experimenting.

So I’ll continue to explore RoR and ponder my deployment options. In the meantime, I’ve come up with a relatively simple way to enhance AustinMania!, still far short of the big vision, but at least a lot more interesting than a static page. Check out the next post for details…

How do I know badbots are visiting? A couple of ways: First, I am using a free service called IP2Location , that shows the general location of a computer based on it’s IP address. According to this, I routinely get visits from the Netherlands, Germany, France, Taiwan, and other “interesting” locales that – one would assume – do not have very many people interested in an Austin specific web site. Hmm, I just got a “visitor” from China, apparently from the FUJIAN region – wherever that is. I have to assume these are robots and not humans. One giveaway is that the server logs show only certain files are requested, and not others that are necessary for human enjoyment of the site. For example, the home page is index.php, but when that page loads in a browser, as it would when a human requests it, a number of other files are required: a javascript file, a stylesheet, and several images. The Chinese visitor, as an example, only requested the root page (index.php) and the javascript file. That’s it - no images or styles. Without them the page would be meaningless.

The other way I know some of these are badbots, and not helpful search engine bots (which I also get a fair share of, btw – Google almost nightly, plus Yahoo, Inktomi, AOL, and others) is that I’ve started to get comment and e-mail spam at my brand-new site. The comment span appeared first – I received 4 comments to various blog postings at the same time, each with what seemed to be just a random collection of words strung together into something that looked kinda like sentences, but really weren’t. Intermixed with the random words were links, also with random words, but the weird thing is at least some of them seemed to point to a legitimate site like news.yahoo.com. I have to admit, I do not understand the purpose of this. Why would a spammer link to a yahoo site? Maybe if I had examined the source code I would have figured out the trick, but I just deleted the comments instead.

Getting protection from comment spam was pretty easy – all I had to do was enable the Akismet plugin, which is very effective (so far) at keeping comment spam away from me. I did have to register at WordPress to get the digital certificate to enable this plugin, but that was pretty painless and only took a few minutes. Hmm, I wonder if there were technical reasons for this, or if WP has stolen a page from the Microsoft playbook. Remember when MS tried to force everyone to get a Passport account? Supposedly you could not get anything out of MS web sites (and even some software) without a Passport. Thankfully that didn’t last very long, and although I managed to get by without a Passport, I now have an unused blog over at WordPress, just to keep spam out of my comment queue. Oh well.

Combating the other scourge, spam e-mail, took a little more doing. The primary line of defense for this is to keep the badbots from finding your addresses in the first place. I was negligent in this regard when I created the Contact Me page, and posted my 6 addresses in plain sight, both to humans and to robots. I knew I would have to come back and fix this, and now that the spam is here, it is time. Actually, it is a little late, because at least one spammer already knows the addresses. So far, I’ve received only one form of spam, so if I cross my fingers tight enough, maybe he’ll not bother to share them with anyone else.

There are a number of ways to hide e-mail addresses from robots, while making them visible to humans, each with varying levels of visitor convenience. There is a nice summary of these methods here .

Since one of my design objectives is to make this site fun and entertaining, visitor convenience ranks pretty high in priority. This means e-mail addresses need to be actual mailto: links, with text that can be easily copied and pasted if the user prefers to do that. Thus, address hiding methods that require any user effort were not considered, such as an images that looks like text (requires users to manually type in the entire address), or text descriptions (in the form “erwin AT austinmash DOT com”), or deliberate address munging (like erwin@REMOVE.austinmash.com).

I also discounted methods that rely on replacing regular text, like “@”, with the equivalent HTML or HEX character encodings, like @. The theory here is that the badbots go hunting for @ symbols, and if they don’t find one, they go away. However, given the declining cost of bandwidth and CPU cycles, and the fact this is easy to decode, I am not about to assume spammers are not smart enough to also be on the look out for @.

There are more elaborate schemes that rely on encryption, CSS tricks, Forms, or user action before revealing the addresses (as if it was a state secret or something). Most of these have maintenance drawbacks, and are just not worth the effort. Simple yet Effective, is my motto. So, I settled on a Javascript approach.

The basic concept is to have Javascript write the text to the page in chunks, such that any one chunk is not enough for a badbot to get anything useful. The end result on the page, however, is a complete e-mail address with mailto: functionality and easy-to-copy-and-paste text. Also, it can be done in a way that is easy to maintain, and if necessary, can even be generated with server-side code.

There are a lot of examples of this technique on the web, but most take an approach that seems convoluted and inelegant to me, such as this: My e-mail address is <script>document.write(“me”);document.write(“@”);
document.write(“here.com”)</script> I suppose this is fine if this is the only address on the page, but I have 6 addresses on my contact page. So I came up with my own technique, which I think is more flexible and re-usable (although it’s pretty likely this has been done before, I did not run across any examples like this in my research).

First, I declared a number of functions to write certain portions of each address, with the intention of reusing common functions (chunks of text). Then, where I wanted the address to render to the page, I just called the functions in sequence.

This is the first part, the function definitions. I placed this in the <head> portion of the page, but it just as easily could have been in a separate .js file.
<script>
function hrf() {document.write("<a href=mailto:")}
function d() {document.write("@austin")}
function d1() {document.write("mash.com")}
function d2() {document.write(">")}
function erwin() {document.write("erwin")}
function press() {document.write("press")}
function sales() {document.write("sales")}
function h() {document.write("help")}
function s() {document.write("submit")}
function payments() {document.write("payments")}
</script>

Then, each time I need an address on the page, I do something like:
<p>Feel free to send me a note at:
<script>hrf();erwin();d();d1();d2();erwin();d();d1()</script>

Here is another part of the page. Note, I am reusing most of the same functions
<p>If you have any difficulty buying pixels, please use this address:
<script>hrf();sales();d();d1();d2();sales();d();d1()</script>

If you want to see the end result, just see my Contact page.

If at some point I create a new page that also has e-mail addresses, I will probably go ahead and put the function definitions in an external file that both pages can access, and will probably re-name the functions to make them less ambiguous and likely to collide with other functions of the same name. Also, I may go ahead and combine d() and d1() – the badbot already knows my domain name, so there’s probably little gained by trying to hide it.

The biggest drawback to this approach is that it won’t work for visitors that don’t have Javascript, or have it turned off. This was not a big issue for me, because without Javascript, the whole site is pretty useless and probably impossible to read anyway. This is a conscious decision on my part, and I’m ok with it. I mean, Disney can spend millions of dollars on Disney World and have some of the greatest rides in the world, but they can’t do anything about someone who refuses to get on because it may mess up their hair, or whatever. Hey – it’s their loss, right?

Oh OK - for the non-scripters, I added a <noscript> tag that says my address is erwin (at) austinmash (dot) com. There you go, you sit here and watch everyone else have fun.

I don’t know how long it takes for an exposed computer to be eaten up by the piranhas of the internet, but I do know a new web site will start to get nibbles in a few weeks. I hope my new boots are thick enough for the coming onslaught.

Yesterday I felt guilty for spending too much time writing code, today I feel embarrassed for not spending enough. After I implemented my new custom permalink mechanism, I thought I’d share that with the world, or at least the readers of the WordPress support forums, in the hopes that it might be useful to someone else. Well, soon after posting my contribution, I realized I had neglected to take care of RSS feed requests for categories, and feed and trackback links for individual postings. Arghhhh!

And to make it worse, the hit counter on my laptop was moving, meaning people were reading the post – and it was wrong! So much for going to bed – I gotta fix this now!

At least the category feed fix was simple enough: create a new /feed/ subfolder under each category folder, each with the same index.php file, with this code:<?php
$path = dirname(dirname($_SERVER['PHP_SELF']));
$position = strrpos($path,'/') + 1;
$thisDir = substr($path,$position);
header("Location: ../../../redirect.php?cat=$thisDir&feed=rss2"); exit;
?>
The feed and trackback URLs for individual posts were more problematic, mainly because I had set up permalinks with the structure /name-of-post.php (for maximum SEO’ness). For individual post feeds and trackbacks, WordPress appends the folder name to the permalink URL, like so: /name-of-post.php/feed/. What this meant was that I would have to convert all my permalink files to permalink folders, so they could have sub folders. And since I had to do that manually, I might as well get rid of the .php in the folder name, since that would result in a weird-looking URL that might freak some people (or robots) out. This had the further unfortunate effect of rendering the permalink I posted on the forum obsolete. Let that be a lesson for you kids – don’t post in haste.

Curiously, the trackback link now works exactly as it did when I had the default permalinks, which is to say, not at all. At this point, I assume this is a completely different issue.

Ok, each permalink folder now contains two sub-folders: /feed/ and /trackback/. The index.php in /feed/ looks like this: <?php
$path = dirname(dirname($_SERVER['PHP_SELF']));
$position = strrpos($path,'/') + 1;
$thisDir = substr($path,$position);
header("Location: ../../redirect.php?p=$thisDir&feed=rss2"); exit;
?>
Note that this and the category feed index.php code converts all sub-feeds to the rss2 format, regardless of what the requestor asked for. This will probably break some feed readers, but I was already spending way more time on this than I should, and the main blog feed is unaffected. If I really wanted to fix this, I would probably replicate some of the functionality of wp-feed.php into the /feed/index.php files.

The index.php in /trackback/ contains this code: <?php
$path = dirname(dirname($_SERVER['PHP_SELF']));
$position = strrpos($path,'/') + 1;
$thisDir = substr($path,$position);
header("Location: ../../redirect.php?p=$thisDir&trackback=true"); exit;
?>

Finally, the redirect.php file had to be updated to support this new functionality. This is what it looks like now: <?php
require_once('wp-config.php');

header("Location: index.php$URLquery");
?>

Am I happy with this solution? Not entirely - RSS compliance is not 100%, and I’ve got a mess of folders and sub folders in my blog file structure. Indeed, one response to my initial forum posting calls my solution a kludge, and I have to agree. I am hoping someone steps up with a better solution, other than “switch hosts”. I’m all ears !

Boy do I feel guilty. I’ve reverted to my comfort zone, writing code, at the expense of marketing activities. I did make a few phone calls, mind you, but I need to put my salesman hat back on, and really tighten up the chin strap so it doesn’t fall off so easily next time. It’s a weakness of mine: give me a good, worthwhile challenge, and I can’t stop until it’s done. Well, at least I take comfort in that it seems I did solve a nagging problem that plagues some WordPress users.

WordPress allows you to define custom permalink structures for your posts, and this seems to work fine for most people. For example, instead of the URL for a post being in the form: /blog/?p=22, the URL can be /blog/name-of-the-post.php. The second URL is favored by search engines, since it supports the content of the page. However, this functionality relies on the web server using the “mod_rewrite” module to read the “.htaccess” file for instructions on how to perform the necessary redirections, and if you are not running your own web server, you are at the mercy of your web host to provide this functionality. Alas, my host does not, so it seemed I was stuck with ugly URLs.

My research into this problem only revealed there are some very frustrated users out there, and that even with a working mod-rewrite module, getting custom permalinks to work in WP was a hit-or-miss affair. Most of the suggested solutions were of the “reinstall everything” variety. I even ran across several threads where someone discovered they did not have the mod_rewrite module installed, so they switched hosts. Well, I didn’t want to do that, so I came up with my own workaround instead.

The way WP implements permalinks is pretty slick (when it works). Instead of actually creating a complex folder structure for all the categories, archives, and posts, all pages are really served by the same file from the same location, /blog directory/index.php, with a query appended to the URL so it knows what to serve up: a category, an archive, or an individual post. Then, when blog pages are generated, all the category, archive, and post links are written according to the custom permalink structure. So, even if the link to an archive is /blog/2006/03/, there really is no such directory, but the link works anyway because the server itself redirects the request back to /index.php, according to the instructions in the .htaccess file. One of the big advantages to this scheme is that the permalink structure can be changed at any time, and there is no need to go and re-name or re-shuffle a bunch of directories and files when you do so. The only things that change are the links on the pages, and the .htaccess file.

Ok, I could not use an .htaccess file, so I was pretty much forced to do the redirections myself. This meant actually creating the nested category and archive folders, and placing a single index.php file in each of them, that redirected the request back to the base folder, to a new file I created, that did the final redirect to index.php. Also, since the custom permalink structure I wanted was: /%postname%.php, I also had to create a file for each post, each one named name-of-post.php that also redirected requests to my new redirect.php file, so it could do the final redirect to index.php.

This sounds complicated, but it’s not that bad, and once it is set up, there is little maintenance involved. Obviously the biggest drawback is that I have to remember to create a new name-of-post.php file every time I make a new post. Also, I need to know the cleaned-up post name that WP will use for the permalink. This could probably be handled with a plug-in, but I’ll leave that as an exercise for the interested reader. For now, I’ll make theses file manually. All I have to do is copy and rename an existing one, so it’s not that bad. The same goes for any new categories I create in the future.

Another drawback to this implementation is that the final URL the user sees is the ugly one, without my custom permalink. I guess I’ll have to live with that, and since this was mostly done to appease the Google God, it should be ok. I did read that the preferred redirection for search engines is a “301 – permanent move”, but according to the RFC 2616 - Hypertext Transfer Protocol — HTTP/1.1 documentation, this will cause the requester to update the URL that was used to the new, redirected one, which would negate any benefit from the custom URL. Instead, I opted to use the default “302 – found” header that PHP sends with the redirect, that signals to the requester that this is a temporary move that might change, so please continue using the original URL. I hope this works.

Do you suppose the Googleites went to the trouble of seeing how long a “temporary” redirect is in effect, and penalize those who use it too long? Should I cower in fear, afraid that gGod might see my impudence, and banish me from his holy index? Whatever. – Hey, if that happens, I’ll put out a press release and get all sorts of attention, right?

Will this be my last tech post? Hardly – there is plenty to do on the site still – at least 4 of the remaining to-do items are tech, and that’s even before I get into expanding AustinMania! But, I think I will be unhappy with myself I don’t get out there and sell more ads. Pixels, anyone?

This is the index.php code that went in every category folder - /blog/categories/austin/index.php, for example:<?php
$path = dirname($_SERVER['PHP_SELF']);
$position = strrpos($path,'/') + 1;
$thisDir = substr($path,$position);
header("Location: ../../redirect.php?cat=$thisDir"); exit;
?>


This is the index.php code that went in every archive folder - /blog/2006/03/index.php, for example:<?php
$path = dirname($_SERVER['PHP_SELF']);
$position = strrpos($path,'/') + 1;
$thisDir = substr($path,$position);
header("Location: ../../redirect.php?m=2006$thisDir"); exit;
?>

Every year, I’ll have to copy the previous year’s folder set, and edit this file to change the year on the last line.

This is the contents of every “name-of-post.php” file:<?php
$path = $_SERVER['PHP_SELF'];
$position = strrpos($path,'/') + 1;
$thisPost = substr($path,$position,-4);
header("Location: redirect.php?p=$thisPost"); exit;
?>


This is the redirect.php file code that receives all the redirects above:<?php
require_once('wp-config.php');

header("Location: index.php$URLquery");
?>


Oh, I know the server’s access logs allow you to see where people are coming from, so if I was so inclined, I could spend time analyzing them, and trying to correlate activity X with an increase in referrer A, but frankly, all I care about are the numbers on my little real-time stat counter on my laptop screen (and now, also on my cell phone screen – check out the PixelList page). Besides, such an analysis requires either the assumption that all activities have an immediate effect, or that the effect delay is known for each activity. Suppose you also did activity Y, and it’s effect delay was shorter than activity X, you could easily come to erroneous conclusions about causality.

If you become a slave to analysis, you will find yourself delaying activity Y until you know the effects of activity X, which is fine if your goal is to become an expert in these matters. If instead, like me, all you want is traffic, you’ll do X, Y, and Z right now, and maybe X again, and try Y.y for the heck of it, and start doing research on alpha and beta. Did Z work better than Y? Who cares? It didn’t hurt, and if traffic went up, so did my smile!

Without further ado, I got most of my SEO ideas from this excellent post by Mark Bloomfield, and I added some tweaks of my own.

The name of the game in SEO seems to be “appeasing the Google God”. Like any good god, gGod is awesomely powerful, with unknowable motives, mysterious ways, and cult-like followers who’s job it is to tell the rest of us how to write virtuous code, so that maybe, if gGod smiles upon us, we too may achieve the highest levels of search results nirvana.

Ok, maybe cult-like is too strong a phrase, but some of the parallels between gGod and Zeus are pretty striking, don’t you think? Zeus had his thunder and lightning, and gGod has his search index, which he can use to smite those who displease him . While Zeus feasted on sacrificial lambs, gGod gets his pleadings via e-mail :
 

I-Newswire owner Eric Borgos:
“I e-mailed Google today to let them know about these changes, so hopefully they will add my site back,”

Good luck, Eric!

As befits the information age, gGod seems to be fixated on words (good thing, I’m fresh out of lambs), and the consistency of the “message” is as important as the sincerity of the sacrifices of yore. For a proper sacrifice, you know, the lamb itself was not enough. If you really wanted things to go your way, you had to make a sturdy pyre to set your offering on, start a good, roaring fire, and make an impassioned speech. Wailing like a baby probably didn’t hurt, either, especially if others joined you. Likewise, the content of a web page is not enough. For gGod to notice you, you need to set your prose at a sturdy URL (with words that support your content, while making a short directory stack), engulf the text with clean code, and expound on your content with relevant meta keywords, page title, and description. Wailing like a baby probably doesn’t hurt, either, especially if others joined to you with hyperlinks.

I was able to implement most of these concepts in this WordPress blog. If you are reading this on my web site (and not through an RSS feed), take a glance at the page title at the top of your browser. You will notice either the blog name and description, category name, or post title in the page title, depending on how you accessed this post (on the blog home page, as part of a category, or as a single post). This text is generated automatically for every page, it is not hard-coded for each post. Also, if you view the source code for the page, you will see the same title text at the beginning of the keywords meta tag, and the first few words of the post excerpt (if it exists; older posts without them will have the first words of the actual post) in the description meta tag. All of this was done to surround the post contents with a relevant “envelope” of data that is consistent with the topic of the post. Supposedly, gGod looks favorably upon this.

A related technique is to put the title of the post in the permalink URL. I have not been able to implement this yet, because my web host does not allow me access to the .htaccess file that is necessary to implement the redirects to make this work. I hope to be able to figure out a workaround to this soon, but I cannot spend too much time on it at the expense of other activities that may bear more fruit. I hope this does not displease gGod too much.

As far as surrounding the page content with clean code, the objective here is to get the words of the post nearer the top of the page HTML. So, I moved the Skyline area map code and related <Form> to the bottom of the page, by moving the PHP that generates this from the header template to the footer template. I also did the same with the call to generate the sidebar, it is now in the footer as well. The result of these efforts is that all the words gGod cares about are consistent in their meaning, which increases relevancy.

The changes to the blog page structures were accomplished by implementing these plugins:

Head META Description
Optimal Title

I also tweaked the title code like so:

<title><?php
if (!$Prod) print "Local ";
optimal_title(' @ ',TRUE);
bloginfo('name');
if (!optimal_title('', FALSE)) { print " > "; bloginfo('description'); }?></title>


The first line of this code prepends the word “Local” to the page title when it is running on my laptop. I got tired of not knowing which of my many open browser windows had my local test and development web site, and which had the public “production” web site. The next two lines just use the Optimal Title plugin and standard bloginfo function, and the last line adds the post description if the plugin is not available for some reason.

Will any of this make any difference in my traffic? Who knows? At the least, I’ve learned something new, I’ve experienced implementing it, and it can’t help but increase my blog rankings in the future. The journey continues, and these side trips make it that much more interesting. Onward ho!

I did consider building my own blogging system, which would have guaranteed maximum flexibility in presentation and functionality, but at the significant cost of delaying implementation and distracting me from the marketing tasks at hand. Having sold an ad, I’m hooked, and I want more, but I feel I need to be careful in how I go about the whole marketing and promotion process, which means I can’t spend all my time writing code.

I did some research on the blogging tools available on my web host, Movable Type and WordPress, by directly looking for opinions of one vs the other. I googled the matchup, found some lame writing, and finally found this post by Dustin Diaz, which was informative and to the point. Most of the (coherent) opinions I found on the web were in favor of WordPress, and I was inclined to go that way myself, but the real clincher was that I found out WordPress is in PHP, while Movable Type is in Perl. Since the rest of this site is in PHP, and I needed to fully integrate blog pages into the site (or, more technically correct, is that I needed to integrate certain site structures, such as the Skyline, into every blog page) there was really only one choice – WordPress.

As you can see, I have managed to tweak the WP templates to pretty much follow the layout and style convention of the rest of the site. This took some doing, actually, since the author of the theme I selected as a starting point likes to place start and end tag pairs across template files (which makes for great fun troubleshooting the HTML structure), and the default XHTML doctype declaration gave me all sorts of grief until I decided to replace it with my tried and true HTML declaration. All in all, though, I am very happy with my ability to bend this system to my will, and it sure beat doing it all myself!

There may be some disagreement now over the “correct” way to code web pages, but the reality is that anyone who aims to make a web site accessible to a wide audience will make sure it works correctly in Internet Explorer first, then may take the time to see that is works ok in other browsers, depending on their intended audience. One way to ensure compatibility is to limit the user experience to reading and navigating, and keeping the interaction and page layout as simple as possible. I could’ve taken that route, but that’s not the way of this site.

It has been interesting navigating the ins and outs of standards-compliant scripting and styling, at least as implemented by Mozilla. A welcome departure from the past was that by and large, changes made to accommodate Firefox also worked in IE, so code branching was kept to a minimum. Oh, there was still the occasional quirk, like brand F seems to include border thickness in an element’s width, while IE does not, but that can be handled with a global adjustment factor - see? I knew fudge factors would come into play at some point (see Jan 7th blog entry)

I also took this opportunity to get my feet wet with AJAX for the real-time statistics on the PixelList page. Well, I say that only because this is my first use of dynamic XML in a non-IE browser, so I’m using the XMLHttpRequest object native to Firefox. As it turns out, I’ve been using Javascript and XML (the JAX in AJAX) in IE for several years now, as a way to dynamically render sections of a page, so this is not all that new to me. Anyway, as the Asynchronous nature of AJAX (yes, that’s the first A) is not really necessary in this application (the content of the page has already loaded, it’s no big deal to wait for the server response with the stats XML), I think I may keep things simple and just use DJAX. Sorry, I don’t tend to go around swinging a sledgehammer to tap in a finishing nail, but at least I’ve got a feel for the tool, and it’s right there in my box in case I get the urge to build something more substantial.

Well, at least some of my goals for this site are being reached. Now I need some ads… Not to worry, the marketing machine has been working in the background, and may bear fruit in a week or so. Stay tuned…

At first glance, this effort seems pretty straightforward: as the potential pixel purchaser expands the grid parcel they want, check to see if there is an existing ad in the way, both in the horizontal and vertical directions, and if so, prevent the new grid parcel from expanding under it in that direction. So conceptually, I just need to know all the x coords and y coords of the upper left corner of each existing ad, to compare them against the coords of the user’s mouse. Since the ad locations are already being generated by PHP code accessing the ad data in the database, it is fairly simple to construct a couple of JavaScript arrays to hold the X and Y coords.

After working this code into the page, it quickly becomes apparent I also need to know the height and width of each ad, since the purchaser will very likely not start his ad on the same grid row or column as the “collision” ad I am trying to avoid. If he starts one row below the top of an ad that is to the right, then I need to know the height of the ad to see if it is in the way

In order to keep the x coords and the heights, and the y coords and widths, grouped together in the arrays, I set them up so that each item in each array is actually a new, 2 item array, with for example the x coord of ad 1, and the height of ad 1, as a small array, which is the first item in the X coords (horizontal) array.

This way, as the code traverses the horizontal and vertical coordinate arrays looking for the next potential ad collision, when it finds it, it will also have the “thickness” of the ad, and so can calculate if there really is a collision or not.

Ok, so I get this working, and it seems to do ok, although the new ad selection rectangle does not behave as smoothly as I would like. No problem, I can tweak it later, but first let’s see how it behaves when there are multiple existing ads on the page, not just this one test ad.

Consider this scenario:

With multiple ads scattered around the page, the code runs into an ordering problem. In order to determine which is the next ad in either direction, the coord arrays need to first be sorted in ascending order. Well, the horizontal sort for these three ads is 2-3-1, while the vertical sort is 1-2-3. So, I ended up with the selection rectangle being limited by the horizontal position of one ad, and the vertical position of another! That ain’t gonna fly…time to step back and rethink my approach to the problem - again.

In case you are wondering, let me explain why all the effort. It starts with the CEO voice, who says: Maximize Revenue! The President voice then echoes: Optimize the Pixel Sales Process! The Engineering VP voice hears this, confers with the Marketing voice, and decrees: Develop a streamlined pixel purchase system - make it simple, fast, and accurate - strive for zero re-work! Naturally, the Product Manager voice then puts in his two cents: Allow Skyline parcel purchase from any page, enable grid ad selection directly on the home page, minimize clicks, maximize accuracy, make it fun and easy! So here I was, the guy actually doing the work, trying to meet requirements. Luckily these particular voices are all in my head, so they normally sing in a smooth chorus, but at times like this they can sound like so much heavy metal.

The codemonger in me warns me to go back, forget it, don’t even try. But the Furion in me hopes I won’t listen…

So I soldier on. After a bit of reflection, it becomes clear I only need one array, with each item in it holding all 4 values of interest for each ad: x coord, y coord, width, and height. And, after futzing around with the previous versions of the collision detection code, I now came up with a wonderfully elegant solution that only takes 2 lines of code in one IF statement, inside of a single FOR loop for each axis, as opposed to the complicated nested mess I had before. I love it when that happens - that’s how I know the solution is right!

Theoretically, this new algorithm should now work in all ad placement scenarios, but I can’t help but still hear the Legal voice muttering something about liabilities, so the “overlapping pixels” disclaimer remains on the buypixels page, for now…

Getting the PHP and MySQL working on the host’s environment took a bit of work. It turns out that the version of MySQL they use, 4.x, does not support Views, and I had set up a few of them in my local environment. I had already checked and discovered that since the host did not support mysqli functions, I knew stored procedures were out, but no Views either? Geez, there go my efforts at creating a scalable system. Ok, maybe it’s not that bad, and it is not that big a deal to convert the View sql to PHP code, but it seemed like a step backward. Time and traffic will tell how this performs in real life.

While I was mucking around in the online version of phpMyAdmin (the MySQL administrator they provide), I noticed some options that are not present in the desktop version of MySQL administrator for version 5.x that I downloaded, such as the apparent ability to safely handle special characters in user text box inputs. Wow, freedom from fear of user hacks - without the usual massaging code - what a concept! I definitely need to experiment with this sometime.

I also took the time to implement the hit, visitor, and ad click counting routines. This task was lower in priority, but then I thought, why cheat myself? Early on, I can use all the hits I get, plus it will be fun to see the numbers climb - assuming they do, of course. I must admit I’m a bit nervous. What if I build it, and no one comes? Anyway, I am aware that my web host has site statistics, and gives me access to the raw logs, but I thought it would be interesting to compare that to data I log myself. Plus, having my own data gives me flexibility in how I can view it.

So the site is now live, but not yet “launched”. I need to put my marketing hat on now. This just gets funner and funner!

My current thinking is to finish the FAQ, do the PixelList page structure (again, don’t need the actual list until someone buys some pixels), and a basic Contact page, then move everything over to the web host and see if it works. Bugger! I still need to figure out how to use the MySQL dump commands to get the Skyline area map data uploaded. No rest for the weary!

Right after launch, the top priority will be modifying the ad-picking code to keep people from buying pixels that overlap existing ads. Then, I need to get the page and click counters running, then I can turn towards displaying the stats on the site. After that, the customer login, reporting, and account editing pages, I think, followed closely by the weekly report e-mail functionality. Man I hope that actually gets used (as in I hope there are customers to e-mail). Along the way, I will be improving the code compatibility with Opera and Firefox, since it’s taking long enough just to get it to work in IE. Interestingly, it seems Opera is closer to IE in how it handles Javascript than Firefox is. I suspect that is because Firefox follows the W3C specification more rigidly.

Later on, I’ll make this page into a proper blog instead of a static HTML page, with the ability for readers to leave comments. Then, after all that, then I get to start adding the fun stuff! Well, as the saying goes, anyplace on Earth is within walking distance, if you have the time…

Using PHP is turning out to be fairly smooth. I can use a lot of my ASP techniques, but with some added twists. For example, there is interesting built-in MySQL functionality, and a variety of ways to assemble text strings and write to the page. After seeing the kinds of traffic spikes experienced by the Million Dollar Home Page, I am mindful of writing efficient code, although that will probably evolve as I learn more about how this platform actually works. I wonder if there is any kind of execution tracing capability, like .Net has, to troubleshoot bottlenecks. Something to research later - I’ve got a site to launch!

My focus now is on getting the ad-buying process mapped out and working. I also need to start doing research on how to use PayPal.