In-memory Javascript Database

I wrote an in-memory Javascript database for a project at work. “What?!” you say? You heard me. In-memory…Javascript…database. Yes, there is a use-case for it.

So our current HTML5 project has moved back its target coverage to IE8. Darn. No CSS3, standard event model, or implicit getters/setters (meaning no data-binding). But it still has localStorage and sessionStorage, the ~5MB data store that an HTML5 app can use. So I wrote a JS database which is really just an array of objects with some easy API for filtering, sorting, limiting, and finding the object you need. Now I can store a bunch of data in memory, getting only the newest content when I poll the server, and much more quickly sort, filter, etc. the data for the user without round-trips to the server. And then when they close the app, I can JSONify the data in the localStorage and load their app much more quickly when the come back. They’ll immediately see where they left off, then when the first server poll finishes they’ll see they’ve got unread items that they can scroll up to view.

I found taffydb.com already existed, but I sure hate unreadable code, and I thought how I could make it faster using the built-in Array filter() method. So I built my own. I also added a full-text search index that can be used for finding based off of keyword matching. I’m working on refactoring for a different query API that supports ORs and sub-expressions–or expressions within parenthesis (e.g. name == “Jacob” and (age == null or age > 10) and cool == true).

Follow the progress of JaDE (Javascript Database Engine) on github.

Reintroducing – Flight Stealth!

Flight Stealth was a Flash component framework a year ago. It went away as the creators of Stealth and OpenFlux combined to create Reflex last year. But Stealth is back! Reflex is forked.

There has been some disappointment that Reflex has forked. Many people have ignorantly but understandably attributed the split to Tyler, the project lead over Flight Stealth. However, Ben Stucki was the one who wanted to fork. After deciding to fork and leave Reflex he asked Tyler if he could keep the Reflex name. Tyler graciously consented, deciding to use the old Stealth name, but since the only difference in a new fork is the projects’ names, he’s gotten the short end of the stick, losing mind share, and for those who were against the fork, getting the blame.

I believe both projects will move forward as a result of this fork. Their project leads each have a vision and direction in mind, and I’m seeing the difference in goals will lead to different products and needs addressed by the projects. Although as Flash developers who currently only have one robust component set in our tool belts, it is hard to imagine the need for more than one alternative. However I believe we will find that one-size-fits-all is not an ideal goal and a specialized framework will always win out over a generic one in its own problem domain.

Good luck to both projects and make us proud.

As an aside, I designed the flightxd.com website where Flight Stealth will be hosted. I am quite proud of what I did, not being a designer.

Please join the Stealth mailing list and find out how you can help. We have a website to put together, user docs that need to be written, components that need to be written, and much much more.

Type overloading in Javascript

One of the core aspects to the Simpli5 Javascript library is that the components you build don’t wrap DOM elements. They actually are DOM elements. This is accomplished using what I call type overloading.

How it works is you take an HTML element in the DOM and make it a different type. It is important–or maybe just ethical ;) –that this new type is a subclass of the original element type. You don’t want to find out that a div is no longer a div, and Firefox has issues with this anyway. So how do you type overload? First create your new type.

function MyDiv() {
    this.addClass('my-div');
}

Then it needs to extend the HTMLDivElement type. There is a trick to this using a dummy type.

function dummy(){}
dummy.prototype = HTMLDivElement.prototype;
MyDiv.prototype = new dummy();

Then we can add methods to the MyDiv.prototype which we may later call.

MyDiv.prototype.addClass = function(className) {
    var classes = this.className ? this.className.split(/\s+/) : [];
    if (classes.indexOf(className) == -1) classes.push(className);
    this.className = classes.join(' ');
};

Now we have our new type. We can’t just create it because to get an HTML element you have to go through document.createElement(). We can either do that or grab an element already in the DOM. Let’s do the latter.

var div = document.getElementsByTagName('div')[0];

Now we have our div. The next part is type overloading. On every Javascript object is a special property “__proto__” (two underscores on both sides) which points to the object’s prototype. It also has “constructor” which points to its type function. We just have to replace these two with our own, changing the type of the object at runtime.

div.__proto__ = MyDiv.prototype;
div.constructor = MyDiv;
div.constructor(); // call the constructor for any initialization that needs to happen.

We now have a div which is of type MyDiv. The following code all returns true and works.

alert(div instanceof HTMLDivElement); // true since we subclassed it
alert(div instanceof MyDiv); // also true since this is our new type
div.addClass('foo'); // works, no errors, the method exists on div
alert(div.className); // "my-div foo" because our constructor added my-div and we just added foo

It’s a really neat thing that you can make objects become new types at runtime. And since every HTML5 browser supported it at the time I wrote Simpli5 it seemed like a great solution. Unfortunately, not all browsers support the __proto__ property. Now that IE9 is publicly available in beta I’ve given it a look. It does not support __proto__. After all, __proto__ isn’t a standardized property. So for Simpli5 I’ll have to for-i-in copy all the methods of my new type over to the instance of an element. It will be slower, and “div instanceof MyDiv” won’t return true, but it should still work for our needs.

Hey Microsoft, I know it isn’t a standard property, but do you think you could make the __proto__ object a publicly accessible read/write property? I would really appreciate it!

Runtime Performance with CSS3 vs Images

I’m pretty happy with the great stuff CSS3 (and HTML5) brings. However, some care should be taken in balancing how many images you load versus the load you put on the CSS engine. And there are a lot of articles on the web encouraging use of the new CSS features such as gradients and shadows in order to optimize for images in your page. But that’s only half the story.

Image Optimization

CSS3 allows you to add drop shadows to your elements, gradients as their backgrounds, and rounded corners on their… corners. Using these few capabilities (you might throw in a couple more like custom fonts and you can put together much of the web’s design with only a few icons needed for images. This allows for much smaller page download because the definition for a shadow or gradient is only a few bytes but an image of these same things are usually kilobytes larger. Pages download much more quickly.

CSS Optimization

While you can do lots of great things with CSS3, drawing shadows and gradients dynamically can affect responsiveness in your page. If you find your page is not scrolling smoothly or dynamic pieces don’t pop like you’d want them to, you might want to optimize your CSS and use more images. Your page may download slower, but once it’s there it will be more responsive.

Case-study on Page Performance

I had a page with a lot of these gradients and shadows (see the previous picture, the original version was 100% CSS, it had no images at all), however, scrolling the page left and right was very clunky and unresponsive. I thought perhaps I had too many HTML elements on the page, but I’ve seen much more work better. After playing around with code a bit it occurred to me that the dynamic calculations and drawing of the gradient’s and shadows was affecting performance. This should have been more apparent to me since it is a common optimization in Flash when you use too much drawing API. After removing the shadows and gradients from my stylesheet the scrolling was smooth again, just like I would expect it to be. Removing the shadow helped out a lot more than removing the gradients. I theorized that the browser may have a better time layering images than it does calculating shadows and gradients, so I tested it out.

After replacing all the gradients and shadows with images, I found my page still scrolled smoothly even though the same shadows and gradients caused it problems with CSS. For my particular case, I am creating a web application which users will come to and stay at for awhile. There is a lot of elements on the page, a lot of design parts to it, and in this particular developer-art incarnation of it, a lot of shadows.

On a side note, the process to replacing the CSS shadows etc. with images was much less painful with CSS3. I didn’t have to alter any HTML because you can layer backgrounds onto an element now so my elements had- background: url(topimage.png) no-repeat left top, url(bottomimage.png) left bottom, url(middleimage.png) repeat-y left; So even though I was forgoing CSS3 shadows it still made my life easier and my page simpler with the images.

The Right Balance

For many web pages out there adding a few shadows or gradients to the page will help make your page look that much nicer and doing it in CSS3 is easier to tweak when you don’t have to re-export images from your site-design-file. But if you have performance problems in your page, you might try using images for some of the heavily repeated elements or the shadows in your page.

My First 10k, Trailer Park Style

I am finally getting around to sharing my memorial day adventure. I ran my first 10k here in Boulder (called the BolderBOULDER). It was pretty fun, considering I wasn’t in great shape for it. I outdid my expectations (I’m slow) with a 68 minute time. Our Jive team running the BolderBOULDER (voted best 10k in the country) planned out a sweet running uniform: jean cutoffs, wife-beater, Jive tattoo, fake mustache, and anything else that screamed “TRAILER PARK!”

UNFORTUNATELY, I was the ONLY one who really followed through on the whole outfit. Guess that means I’m totally the winner in the style category. Here are some photos to commemorate the event.

Starting out

Starting out

Just gettin’ going

Getting cooled off

Breaking a mile barrier

Home stretch

After Pose (mustache finally fell off)

Developers Put Their Heads in the Sand

As developers, we like to put our heads in the sand. We’d be much more successful if we didn’t. Let me explain.

When I first learned about basic object oriented programming, I was suddenly disgusted with functions and code that wasn’t an object. I got over it.

When I learned about composition over inheritance, that became the standard by which I judged all code, mine included. It became my fixation. I got over it.

When I learned about design patterns, I wanted to apply them to every situation, and I wanted to do it right and apply them exactly the way prescribed in the pattern. I got over it.

When I learned about optimizing code, I spat on for loops that didn’t initialize the length first, I kicked dirt around static methods which are shown to be slower (in the language I was using) than instance methods, and I generally despised any type of code which I read on a blog was slower than an optimized alternative. I got over it.

I could go on with dependency injection, abstraction, database normalization, and on, and on, and on. I hope I continue to get over over-applying new knowledge.

I’ve always wanted to iterate over and rewrite again and again pieces of code, reusable libraries, and other gleaming nuggets I’ve done in the past which could be more perfect. Even faster. Even better. I’m getting over it.

In the last few months I’ve been working with some very bright and pragmatic developers. They’ve been teaching me, unbeknownst to themselves, to look at the big picture and to get my freaking head out of the sand.

Just because instance methods are slower than statics doesn’t mean you shouldn’t use them. Creating an object is slower than calling a method, but should we throw OOP out the window and stick with functional programming? Messing up the Flyweight pattern (or even heaven forbid, our blessed MVC pattern) by altering it from the original and all knowing gang-of-four specification doesn’t end-of-life your product before it’s out the door. What keeps your product from releasing is rewriting it, or pieces of it, over and over again. If it works, DON’T FIX IT. Only optimize if your application is too slow for your users. And then only optimize the slowest parts. Next application you write you can do it better, but freaking finish! Clean up and refactor as you add new features for your users. Don’t waste time redoing anything from scratch unless its complete junk. And if you wrote it, figure out why you’re writing complete junk in the first place and fix the root of the problem.

I’m writing a library in Javascript for HTML5 applications that I know will make many Javascript developer weep. It flys in the face of all the wisdom and standards that they’ve read about on their favorite blogs. But it will make developing applications easier and more maintainable. I know the rules, and thus I know how and when to break them because I know why they were established.

I’m still watching the development of Reflex, the Flash component library my brother is involved in (I’m not actively participating anymore). They’re fighting against head-sand-itis both externally and internally. Project supporters may disagree with things proposed because it’s not right, or it’s not as fast as machine code. Internally with themselves they’re disgusted at breaking precious programming rules. Finally they’re starting to let ideals slide because progression is so slow. Eventually, I hope, they’ll raise their sites to the end goal and do what is necessary to reach it. Even if it means using brains instead of rules. (no offense to any individual person or the general group, we all struggle with it, we’re developers)

The rules are generalities, guidelines to help us until we understand the principle behind the rule well enough to make our own decisions. Understand your natural tendency to over-apply and work against your nature to be more pragmatic.

May we all work on seeing the big picture, understanding the principle behind the rule, and creating great experiences for those who will use our applications. I’m certainly the pot calling the kettle black on this.

« Previous PageNext Page »